Tool & Extension Layout: Manifests, Resolution, And Fallbacks

Introduction to Distribution Tool and Extension Management

Managing software distributions and extensions effectively is crucial for any modern development tool. The dist/tool/extension layout forms the backbone of this management system, dictating how different versions and platform-specific components are organized and accessed. When we talk about distribution tool and extension layout design, we're essentially defining the blueprint for a robust and scalable system. This involves a deep dive into folder structures, manifest schemas, and how the system resolves the correct components for a given environment. A well-defined layout ensures that developers can easily integrate, update, and manage the various tools and extensions their projects depend on, without encountering version conflicts or platform-specific issues. The goal is to create a system that is both flexible and predictable, allowing for global and local configurations to coexist harmoniously. This design document aims to outline a clear structure for this management system, focusing on the essential elements that make it work seamlessly.

We'll be exploring the nuances of folder organization, including the distinction between global installations and project-local overrides. This is key to supporting diverse development workflows, where some users might prefer a centralized management of tools, while others need isolated environments for specific projects. The naming conventions for RIDs (Runtime Identifiers), such as linux-x64, osx-arm64, and win-x64, are vital for ensuring that the correct binaries and assets are downloaded and used. A consistent and well-documented RID strategy prevents compatibility problems and simplifies the resolution process. Furthermore, understanding the precedence rules—how the system decides which configuration or component to use when multiple options are available—is paramount. This layered approach, which we'll align with existing layered configuration principles, ensures that project-specific needs can override global defaults, providing a powerful mechanism for customization. The definition of manifest schemas, particularly using JSONC (JSON with Comments), is another cornerstone. These manifests will act as metadata repositories, containing essential information like the name, version, platform compatibility, source URL, and integrity checks for each tool or extension. The choice of JSONC is deliberate, allowing for human-readable comments within the configuration files, which aids in debugging and understanding.

Finally, this design will detail the config keys used to manage these selections and download locations, as well as establish a clear fallback policy. This policy addresses what happens when a specific local configuration is not found—should the system default to a global setting, or should it throw an error? Making these decisions upfront and documenting them clearly is essential for user experience and system stability. The outcome of this design process will be a concise design document, coupled with necessary updates to our Product Requirements Document (PRD) or implementation notes, ensuring that this new management system is well-understood and ready for implementation. The ultimate aim is to create a system that is not only technically sound but also intuitive and user-friendly, empowering developers to manage their toolchains with confidence and efficiency.

Folder Layout: Global Roots vs. Local Overrides

When architecting the dist/tool/extension layout, one of the fundamental decisions is how to handle the scope of installed components. We need to define clear strategies for both global and local roots. A global root typically refers to a system-wide installation directory, accessible by all users or all projects on a machine. This is ideal for tools and extensions that are commonly used and don't have project-specific dependencies or version requirements. For instance, a core build tool or a linter might be installed globally, ensuring consistency across the development environment. This approach reduces redundant downloads and simplifies updates, as changes made to a globally installed component are immediately reflected everywhere. However, it can lead to conflicts if different projects require different versions of the same tool. This is where the concept of local roots becomes critically important. A local root, often residing within a project's directory structure (e.g., in a .tools or .extensions subfolder), allows for project-specific installations. This isolation is invaluable for managing dependencies that might clash with other projects or for using bleeding-edge versions of tools for experimentation. It ensures that a project's build process or runtime environment is self-contained and predictable, unaffected by changes in the global environment or other projects.

The interplay between global and local roots needs to be carefully managed through a defined resolution precedence. The proposed model prioritizes project-local configurations. If a project has opted into using local extensions and specifies particular versions or sources, these will take precedence over any user or global defaults. This provides developers with granular control over their project's dependencies. Following the project-local setting, the system will then check for user-specific configurations, which can provide defaults for a particular user across multiple projects. Finally, if neither project-local nor user-specific settings are found, the system will fall back to a global default. This layered approach—project-local → user/global default—ensures that configurations are applied in the most specific order, respecting developer intent. Each of these layers needs to be distinctly organized within the file system. For example, global installations might reside in a system-managed directory (like /usr/local/share/mytool or ~/.mytool/global), while local installations would be within the project repository. The naming convention for subdirectories within these roots, particularly for platform-specific components, is also crucial. This leads us to the importance of RID naming.

We need a standardized way to identify the target runtime environment. Common RIDs include linux-x64, osx-arm64, win-x64, and potentially more granular ones like linux-musl-x64. This convention ensures that the correct binaries, libraries, or extensions are downloaded and installed for the specific operating system and architecture. For instance, a tool built for macOS on an ARM processor (osx-arm64) will have different binaries than one built for Linux on an x64 architecture (linux-x64). The folder structure within the roots would likely reflect these RIDs, e.g., $ROOT/mytool/1.2.3/linux-x64/ or $ROOT/extensions/myextension/v2/win-x64/. This clear, hierarchical structure makes it easy for the resolution mechanism to locate the appropriate files. By defining these distinct locations and their resolution hierarchy, we establish a robust foundation for managing diverse tool and extension requirements within a development ecosystem, accommodating both centralized and decentralized management strategies effectively.

Manifest Schema: JSONC for Clarity and Detail

To effectively manage the distribution and resolution of tools and extensions, each component needs a descriptive metadata file. We propose using JSONC (JSON with Comments) for our manifest schema. The choice of JSONC over plain JSON is deliberate; it allows developers and system administrators to embed comments directly within the manifest files. This greatly enhances readability and maintainability, making it easier to understand the purpose of each field, the rationale behind specific version choices, or any custom configurations. Plain JSON can be cryptic, especially for complex configurations, and the ability to add inline explanations in JSONC significantly reduces the learning curve and aids in debugging. The .json extension will be used for these files, even though they support comments, aligning with common practices for JSON-like configuration files. This consistency ensures broader compatibility with existing tools that might parse .json files, while still leveraging the added benefits of comments.

Each manifest file will contain essential information about a specific tool or extension. Key fields within the schema will include:

• name: A unique identifier for the tool or extension (e.g., "my-linter", "code-formatter"). This is crucial for referencing and managing components.
• version: The specific version number of the component (e.g., "1.5.2"). This allows for precise version control and dependency management.
• platform: An array or string specifying the target RIDs for which this component is intended (e.g., ["linux-x64", "win-x64"] or a general "any"). This ensures that only compatible components are considered for download and installation.
• sourceUrl: The URL from which the component's package (e.g., a zip archive, tarball, or executable) can be downloaded. This is the primary location for obtaining the component's assets.
• hash (optional but recommended): A checksum (e.g., SHA256) of the downloaded file. This is vital for minimal integrity check requirements, ensuring that the downloaded file has not been corrupted or tampered with during transit. This adds a critical layer of security and reliability.
• metadata (optional): An object for storing any additional, arbitrary information relevant to the component, such as author details, a description, licensing information, or specific configuration parameters unique to this version.

The structure of these manifests will be aligned with the layered configuration approach, meaning that a project-local manifest might specify a version or source that overrides a global one. The resolution process will read these manifests to determine which component to fetch and verify. For example, if a project needs version "1.5.2" of "my-linter" for linux-x64, its local manifest would point to the appropriate sourceUrl and optionally provide a hash for that specific version and platform. The system would then download the file from the sourceUrl, verify its integrity using the hash, and install it in the project's local root. If a hash is not provided, a decision needs to be made regarding the level of trust placed in the sourceUrl. A minimal integrity check might still be performed, or the system might proceed without one, depending on the security policy and the chosen fallback behavior. The use of JSONC makes these details explicit and understandable, fostering transparency and trust in the distribution management system. This well-defined schema is fundamental to building a reliable and secure mechanism for managing development tools and extensions.

Resolution Precedence and Fallback Policy

Establishing a clear resolution precedence strategy is paramount for the effective management of the dist/tool/extension layout. This strategy dictates the order in which the system looks for and selects the appropriate tool or extension version when multiple options might be available. We aim to align this with existing layered configuration principles, ensuring a predictable and developer-centric experience. The proposed precedence order is as follows:

1. Project-Local: If a project explicitly opts into managing its tools and extensions locally (e.g., via a configuration file within the project repository), the system will first look for the required component in the project's local root. This could be a specific version of a tool or a particular extension package. This layer provides the highest level of control, allowing developers to ensure that their project uses precisely the versions they've tested and approved, regardless of what might be installed globally or for other projects.
2. User/Global Default: If the component is not found in the project-local configuration, or if the project has not opted into local management for that specific component category, the system will then check for a user-defined or global default. A user default might be configured in a user's home directory, providing a common set of tools and extensions across all projects for that user. A global default is typically set at the system level and applies to all users. The system will attempt to resolve the component based on these broader configurations.
3. Error: If, after checking both project-local and user/global defaults, the required tool or extension cannot be resolved, the system will raise an error. This ensures that the user is explicitly informed when a necessary dependency is missing, preventing silent failures or unexpected behavior.

This hierarchical approach ensures that project-specific needs are always met first, followed by user preferences, and finally global standards. This model is applied per category of tool or extension, allowing for flexible configuration where, for example, a build tool might be managed locally, while a documentation generator could rely on a global default.

Complementing the precedence rules is the fallback policy. This policy defines the system's behavior when a local selection is missing or explicitly falls through. We need to make a critical decision here: should the system automatically attempt to fall back to a global or user default if a project-local selection is absent, or should it strictly error out? The proposed approach favors strictness for local selections: if a project explicitly defines a requirement locally, and that specific version or configuration is not found, it should result in an error. This prevents subtle discrepancies between the project's declared state and its actual runtime environment. However, if a project hasn't explicitly defined a local requirement for a category, then the system should fall back to checking user and global defaults. This means:

• If a project opts-in to local management and specifies a component, and it's not found locally -> Error.
• If a project does not opt-in or doesn't specify a component in its local config, the system will check user/global defaults. If found there -> Use it. If not found there -> Error.

This policy balances flexibility with explicitness. It prevents accidental use of incorrect versions due to implicit fallbacks while still allowing projects to leverage shared, global resources when local isolation isn't strictly necessary. The config keys will be essential for managing these selections. We will define specific configuration keys, likely within a central configuration file or environment variables, that users and projects can use to specify: a) whether local management is enabled for certain categories, b) the download locations for global/user defaults, and c) potentially explicit paths or references for user-defined defaults. These keys will guide the resolution engine in navigating the precedence hierarchy. For instance, a config key like tool.extensions.local.enabled=true might trigger the project-local lookup, while tool.extensions.download.path=/path/to/global/tools would specify where global components are stored.

Acceptance Criteria and Documentation

To ensure the successful implementation of our dist/tool/extension layout design, we have defined a clear set of acceptance criteria. These criteria serve as the benchmarks against which the completed design will be evaluated. Primarily, we require an approved design that comprehensively covers the proposed folder layout, including the distinction between global and local roots, and the precise specification of manifest schemas using JSONC. The design must clearly articulate the resolution precedence rules—how project-local settings interact with user and global defaults—and define the necessary config keys for managing these aspects. Crucially, the design must include a well-documented fallback policy, outlining the system's behavior when components are not found at various levels of the hierarchy. This approved design document will form the technical specification for implementation.

Secondly, a key decision point within this design concerns the RID naming convention. We need to finalize and document a consistent and unambiguous strategy for identifying runtime environments (e.g., linux-x64, osx-arm64, win-x64). This convention must be practical, extensible, and align with industry standards where applicable. Alongside the RID naming, the design must also specify the minimal integrity check requirements. This involves defining the acceptable methods for verifying the integrity of downloaded components, such as mandating specific hash algorithms (e.g., SHA256) or outlining any security considerations related to unsigned packages. This ensures that downloaded assets are trustworthy and haven't been compromised.

Finally, the successful completion of this design phase necessitates that notes are added to the PRD (Product Requirements Document) or implementation notes. These notes will serve as the official record of the design decisions, rationale, and any trade-offs made. They should clearly translate the design specifications into actionable requirements for the development team. This includes detailing the structure of the manifest files, the logic for the resolution engine, the configuration options available to users, and the expected behavior of the system under various scenarios. This documentation ensures that all stakeholders—developers, testers, and future maintainers—have a shared understanding of how the tool and extension management system is intended to function. By meeting these acceptance criteria, we can be confident that we have established a robust, scalable, and user-friendly framework for managing development tools and extensions, paving the way for a more streamlined and efficient development workflow.

For further reading on best practices in software distribution and dependency management, you can explore resources from organizations like the Linux Foundation or delve into package management systems like npm or Maven for insights into their design philosophies.