Streamlining Model Types: A Guide To Efficient Architecture

The Core Problem: Dual Maintenance of Model Types

Eliminating dual maintenance is the name of the game when we talk about software architecture. In this scenario, we're focused on how to handle generated model types effectively. The current system, while intending to simplify things, has ironically introduced more complexity. The fundamental issue revolves around the fact that developers are forced to maintain two versions of the same information: models.yaml (the source of truth) and the generated TypeScript files. This duplication leads to potential inconsistencies and, more critically, increases the chance of errors. The goal is to ensure that the process is automated and the source of truth remains singular, making it easier to manage and update. The focus is to make the architecture more streamlined. The architecture needs to be simplified to ensure efficiency. It all starts with the basics of good software engineering and the need to streamline processes.

The Dilemma of Git Tracking

One of the biggest culprits of complexity in the current system is the fact that generated files are tracked by Git. This seems convenient initially, but it quickly becomes a bottleneck. When multiple branches work on changes, it leads to merge conflicts. It requires that developers manually regenerate and commit files after changes to the YAML file, which opens the door to human error. Git tracking goes against a single source of truth and violates core principles. This approach to version control can lead to inefficiencies, and the need for frequent manual intervention is a red flag. The desired solution should automate this, ensuring a smooth build and deployment process without constant intervention.

Build-Time vs. Commit-Time Generation

Another point of concern is the generation timing. Right now, the process is manual, and it relies on developers to run specific commands before committing. The ideal situation is one where the generation happens automatically during the build process. This change eliminates the risk of developers forgetting to run the generation commands and reduces the chances of CI failures. By automating the process, the build becomes more reliable and consistent. The goal is to move from a manual process to an automated one, which results in faster and more reliable builds. This change simplifies the workflow, saves time, and reduces the likelihood of errors.

The Shortcomings of Lefthook Validation

The current Lefthook validation is a good idea in principle. However, it falls short of providing a full solution because it only validates and doesn't automatically regenerate files. This means that even with the validation in place, the system is still prone to errors caused by out-of-sync generated files. The system needs to go further to make the validation process as effective as possible. The aim is to make the entire process as automated as possible. If the files are out of sync, then the build fails, which is not ideal. This leads to wasted time and effort, so the solution is to automatically regenerate files before the commit.

The Dual Maintenance Dilemma

One of the biggest issues is the dual maintenance problem. Developers must maintain both models.yaml and the generated files. This is exactly what the initial goal was to solve, but the current approach exacerbates it. A streamlined solution should treat models.yaml as the single source of truth, with the generated files being automatically updated during the build process. The ultimate aim is to remove the burden of manual maintenance. Developers need to focus on one source of truth.

Current vs. Expected Architecture

Proposed Solution: Option A - Build-Time Only Generation

Option A is the recommended solution and emphasizes simplicity and automation. The generated files are only created during the build process and are not tracked by Git. This method ensures that the YAML file is the single source of truth and eliminates the possibility of merge conflicts. The build process becomes the central point where the types are generated. This approach makes the process cleaner and more efficient. The advantages are numerous, especially when it comes to long-term maintainability.

Step-by-Step Implementation

1. Modify the .gitignore: Add the generated files to the .gitignore file, ensuring they are not tracked by Git.

   # Generated model types - regenerated at build time
   src/tools/config/generated/*.ts
   src/tools/config/generated/README.md
   !src/tools/config/generated/.gitkeep
   

   This ensures that the files are ignored.

2. Adjust the Build Process: The build process must be modified to include a step that regenerates the model types automatically. This often means adding a command to regenerate the types before the code compilation step.

   "build": "npm run generate:models:internal && tsc && npm run copy-yaml"
   

   The command structure ensures that the types are always up to date.

3. Update Imports: Make adjustments to the imports to ensure the system is working properly. The updates make the system more dynamic and flexible, especially if the generated files are not committed to Git.

Advantages of Build-Time Generation

• Simplified Workflow: Eliminates the need for manual intervention.
• Reduced Conflicts: No more merge conflicts related to the generated files.
• Single Source of Truth: models.yaml is the only source.
• Consistent Builds: Ensures that types are always up to date during the build process.

Proposed Solution: Option B - Keep Generated Files But Automate

Option B is a less clean solution, but it still resolves the core issues by automating the generation process. It maintains the current approach, but it ensures that the generated files are automatically updated before each commit. This solution ensures that the system remains consistent and eliminates the risk of outdated type definitions. The automation is the main goal.

Step-by-Step Implementation

1. Pre-Commit Hook: Implement a pre-commit hook that automatically regenerates and stages the generated files. This is typically done using a tool like Lefthook.

   pre-commit:
     commands:
       generate-models:
         run: npm run generate:models && git add src/tools/config/generated/
   

   This hook ensures that the files are always up-to-date.

2. Maintaining the Current Approach: This keeps the existing structure, but it automates the process and resolves the problems caused by out-of-sync type definitions. It keeps the current workflow with automation.

Advantages of Automated Generation

• Automation: Automates the regeneration of files.
• Consistency: The system is consistent.
• Reduced Errors: Lowers the risk of errors.

Acceptance Criteria for the New Architecture

The most important step is to make sure the process is automated. The criteria are:

• Generated files are NOT committed to git: This is a critical step.
• models.yaml is the single source of truth: It avoids duplication.
• Build process generates types automatically: This is key.
• No manual intervention needed: The process needs to be automatic.
• CI passes without requiring manual regeneration: No errors.
• npm run build produces working code from clean checkout: The process is smooth.

Recommended Agents

The following agents are recommended for the implementation process.

• @architecture-advisor: This agent handles architectural decisions, ADR creation, and the evaluation of build-time versus commit-time generation.
• @mcp-tool-builder: This agent implements the chosen approach, either by adjusting the .gitignore or by setting up auto-regeneration.
• @tdd-workflow: This agent verifies that the build works correctly from a clean checkout, ensuring that the changes have the intended effect.
• @documentation-generator: This agent updates the README file with the new workflow, making sure the team understands the process.

Conclusion

By following these steps and implementing the proposed solutions, the team can eliminate dual maintenance, simplify the workflow, and improve the efficiency of the software development process. The key is to automate the process and treat models.yaml as the single source of truth.

For more information and best practices, check out the official documentation.