Tracking API call costs is tough - here's what I built

I'm quite sure there are users here who utilize callin.io for orchestrating workflows that involve various AI tools. You might have scenarios calling OpenAI for text generation, perhaps Anthropic Claude for nuanced responses, or even image/video AI – all automated seamlessly by callin.io, of course.

However, here's the inconvenient part. While callin.io simplifies the automation process, tracking the combined cost of all those different AI API calls within your workflows is far from simple. How much does that complex customer support automation really cost per run, considering every AI module it interacts with? I found myself manually writing scripts to log data or making rough estimations, lacking a clear, unified view.

I was frustrated. So, I developed AIBillingDashboard.com.

So, how does it assist me? Here's what I've observed so far:

1. Unified AI Cost Dashboard: I can now see spending across all the AI services my workflows utilize (OpenAI, Claude, Gemini, RunwayML, Eleven Labs, etc.) in a single, central location.
2. Scenario Cost Insights: By logging usage data from workflows (e.g., using HTTP requests or tools like Google Sheets/Airtable to capture token counts/service calls) and uploading it, I can visualize the AI costs associated with specific scenarios.
3. Optimize AI Spend: I can now identify which AI modules within automations are the most expensive. For instance, could a different model or provider save me more money without compromising the quality of completions/outputs?
4. Centralized Billing: Tracking billing cycles and receiving payment alerts for all the disparate AI services that charge at different times of the month is quite beneficial.

I built this because managing the costs of my own, and others', multi-step AI automations was becoming unmanageable.

How are you currently tracking the AI costs generated within your callin.io workflows? Do you have any clever workarounds or face significant challenges?