Fixing KPI Dashboard Deployment Failure: Missing Tags_json

Encountering issues while deploying your KPI dashboard, specifically with the kpi_tracker due to a missing tags_json column in your summary_view? You're not alone! This article will guide you through resolving this common problem, ensuring your KPI dashboard deployments run smoothly. We'll explore solutions for both adjusting your summary_view and modifying the kpi_tracker query, weighing the pros and cons of each approach. Let's dive in!

Understanding the Problem: The Missing tags_json

The error you're seeing arises because the kpi_tracker query expects resource tags in a specific format: a map of string to string, represented as a tags_json column. Your current summary_view, however, defines resource tags individually, like this:

resource_tags['user_team'] "tag_team",
resource_tags['user_cost_center'] "tag_cost_center",
...

This discrepancy causes the kpi_tracker view creation to fail. The core challenge is to bridge this gap without negatively impacting existing production dashboards that rely on your current summary_view definition. We need to find an efficient and minimally disruptive solution.

Solution 1: Modifying the summary_view to include tags_json

One approach is to alter your summary_view to create the tags_json column. This involves aggregating your individual resource tags into a JSON format. While this might seem complex, Athena provides functions to achieve this. The main advantage of this approach is that it directly addresses the requirement of the kpi_tracker query, ensuring compatibility. By incorporating the tags_json column in the summary view, the kpi_tracker query can seamlessly access and utilize the resource tag data in the expected format. This eliminates the need for modifications to the kpi_tracker query itself, streamlining the deployment process and reducing the risk of introducing errors or inconsistencies. Here's how you can do it using the map_agg function in Athena:

TRANSFORM_VALUES(MAP_AGG(key, value), (k, v) -> CAST(k AS VARCHAR), (k, v) -> CAST(v AS VARCHAR)) AS tags_json

Here are the steps to implement this solution:

1. Incorporate the map_agg function: Modify your summary_view query to include a tags_json column. This column should be created by aggregating your individual resource tags into a map.
2. Use TRANSFORM_VALUES for type casting: Since MAP_AGG can sometimes produce unexpected types, use the TRANSFORM_VALUES function to explicitly cast the keys and values to VARCHAR.
3. Carefully test the changes: Before deploying the modified summary_view to production, thoroughly test the changes in a non-production environment to ensure that the tags_json column is correctly created and that existing dashboards are not affected.

Considerations:

• Impact on existing dashboards: This is the most crucial consideration. Ensure that adding tags_json doesn't break any existing dashboards that rely on the individual tag columns. You might need to update those dashboards to utilize the tags_json column as well, or maintain both formats for a transition period.
• Performance: Aggregating tags into a JSON structure might introduce a slight performance overhead. Monitor your query execution times to ensure that the impact is acceptable.
• Complexity: Modifying the summary_view could potentially increase the complexity of the query. It's important to document the changes and ensure that the query remains maintainable.

By carefully implementing these steps and considering the potential impacts, you can successfully modify the summary_view to include the tags_json column and resolve the deployment failure of the kpi_tracker.

Solution 2: Adapting the kpi_tracker Query to Use Existing Resource Tags

Alternatively, you can adjust the kpi_tracker query to work with your existing resource tag format. This means modifying the query to extract data from the individual tag columns (tag_team, tag_cost_center, etc.) instead of expecting a tags_json column. This approach avoids altering your summary_view, minimizing the risk of disrupting existing production dashboards. However, it requires changes to the kpi_tracker query itself, which might be more complex depending on the query's structure. In adapting the kpi_tracker query to use existing resource tags, you preserve the integrity of your current summary_view and minimize the risk of impacting existing dashboards that rely on the individual tag columns. Here’s how:

1. Modify the kpi_tracker query: Instead of referencing tags_json, extract data directly from the individual tag columns (tag_team, tag_cost_center, etc.).
2. Update filtering and aggregation: Adjust any filtering or aggregation logic in the kpi_tracker query to use the individual tag columns.
3. Test the changes thoroughly: Before deploying the modified kpi_tracker query, thoroughly test the changes in a non-production environment to ensure that the data is correctly extracted and that the KPI dashboard functions as expected.

Example:

If the original kpi_tracker query had a clause like this:

WHERE tags_json['team'] = 'MyTeam'

You would change it to:

WHERE tag_team = 'MyTeam'

Considerations:

• Complexity of kpi_tracker query: If the kpi_tracker query is complex, modifying it to work with individual tag columns might be challenging and error-prone.
• Maintainability: Maintaining a modified version of the kpi_tracker query could be more difficult in the long run, especially if the original query is updated in future versions of the cloud-intelligence-dashboards-framework.
• Duplication of logic: If other queries also need to access resource tags, you might end up duplicating the logic for extracting data from individual tag columns.

By carefully considering these factors and thoroughly testing the changes, you can successfully adapt the kpi_tracker query to use existing resource tags and resolve the deployment failure.

Choosing the Right Solution

Both solutions have their merits and drawbacks. Here's a summary to help you decide which approach is best for your situation:

Recommendation:

If minimizing disruption to existing dashboards is your top priority, adapting the kpi_tracker query might be the better option. However, if you're looking for a more standardized and maintainable solution in the long run, modifying the summary_view to include tags_json is recommended.

Implementing the Chosen Solution

Once you've chosen a solution, follow these steps to implement it:

1. Develop the changes: Implement the necessary modifications to either the summary_view or the kpi_tracker query.
2. Test thoroughly: Test the changes in a non-production environment to ensure that the KPI dashboard functions as expected and that existing dashboards are not affected.
3. Deploy to production: Deploy the changes to your production environment.
4. Monitor performance: Monitor the performance of the KPI dashboard and the underlying queries to ensure that there are no performance issues.

Conclusion

Resolving the tags_json issue requires a thoughtful approach, considering the impact on existing infrastructure and long-term maintainability. By carefully evaluating the two solutions presented and following the implementation steps, you can successfully deploy your KPI dashboard and gain valuable insights from your data.

For more information about Cloud Intelligence Dashboards visit the AWS Cloud Intelligence Dashboards page. There you can find useful information about implementation and how to get started. This article is for informational purposes only. Always test changes in a non-production environment before deploying to production.