FamilyMate.AI - Create AI Agent

Hi All,

After completing the callin.io Academy I wanted to build a real-world project. I decided to build something to help with family life

. I am married with two young kids. My wife and I find ourselves doing weekly planning to organize the upcoming week. We were putting events into Google Calendar

along with a physical calendar. Throughout the week we would be adding new events, updating events, etc. We would be telling each other about updates and also messaging each other about updates. I found myself getting lost and not keeping up.

I built an AI Assistant

to help us manage our shared Google Calendar. It can list, create, update, and delete events. A callin.io scenario can update us every morning with the events for that day.

I used the new callin.io AI Agents module.

Scenarios Overview

Communicating with the AI Assistant

I was going to use WhatsApp but realized that I would need a WhatsApp Business account to get the functionality that I needed. Because of this I used Telegram. Setup was easy; you first create a new bot using the BotFather

You will be given an access token which you will use in your callin.io scenario to configure your Telegram Bot.

AI Agent Tools

I had to create a number of tools for my AI Agent. A tool in this case is simply a callin.io scenario where you define scenario inputs. For example, here is my Update Google Calendar Event scenario

Create Google Calendar Event Tools

I spent a lot of time working on Create Google Calendar Event. At the beginning, I had one scenario. It started to get complex because there are a number of different options when creating the event.

It can be an all-day event where you have to specify the event name and start date. It can be an event where you specify either an end date or duration. I had many scenario inputs to allow it to handle the three use cases. This was causing issues when the AI Agent was calling the scenario because it was getting confused about what scenario inputs it should use for the different use cases.

I solved this by implementing the three use cases in their own scenarios, therefore having their own clearly defined scenario inputs.

Adding Tools to callin.io AI Agent

When my scenarios were finished, tested, and verified as working, I then added them to the callin.io AI Agent using 'Add tools'.

Then you have to add a prompt to Additional system instructions to tell the agent what tools are available and any rules you want it to follow.

You have access to the following tools

Tools:

1. Create All Day Google Calendar Event. Use this to create a new all-day calendar event. Args for the tool are
   eventname (text)
   startdate_string (text)

2. Create Google Calendar Event With Duration. Use this to create a new calendar event with duration. Args for the tool are
   eventname (text)
   startdatestring (text)
   durationstring (text)

3. Create Google Calendar Event with Start and End. Use this to create a new calendar event with start and end. Args for the tool are
   eventname (text)
   startdatestring (text)
   enddate_string (text)

4. Delete Google Calendar Event. Use this to delete a calendar event. Args for the tool are
   event_id (text)

5. Search Google Calendar Event with Start and End. Use this to search calendar events with start and end. Args for the tool are
   startdatestring (text)
   enddatestring (text)

6. Update Google Calendar Event. Use this to update a calendar event. Args for the tool are
   eventname (text)
   eventid (text)
   startdatestring (text)
   enddatestring (text)

Rules:

• Today is {{formatDate(now; "DD.MM.YYYY HH:mm"; "Europe/Prague")}}.
• Start and end date string format is DD.MM.YYYY HH:mm.
• duration_string string format is HH:mm.

Date Issues

I ran into some issues when using Date type on the scenario inputs. My timezone is Europe/Prague, and that is what I have set in my callin.io settings. When implementing the Create Google Calendar Event scenario, I noticed the following:

1. Sent Telegram message asking to create an event 'tomorrow at 1pm'.
2. callin.io AI agent scenario called the Create Event scenario, passing 'tomorrow 1pm' in the send_date scenario input.
3. The Create Event scenario was setting the Start Date as 'tomorrow 3pm'. It was adding 2 hours.

I do not fully understand what was happening. I think that somewhere 'tomorrow at 1pm' was interpreted as UTC time and was getting converted to Europe/Prague time. If anyone can confirm that this is what was happening and explain exactly where the conversion happens, it would be interesting to know

When I ran the scenario on its own, setting the scenario input Send Date, the time was not changed. This makes me think the conversion was happening outside of the scenario

My solution was to change the scenario inputs to senddatestring, enddatestring, etc., and use parseDate in the module mapping.

In the Agent prompt, I included the following:

• Start and end date string format is DD.MM.YYYY HH:mm.

Conclusion

callin.io AI Agents are very powerful. I spent many hours working on this, which I think is to be expected as it was my first time implementing an AI Agent. But now that I have gone through the process, any future AI Agents will be much quicker, and I have a list of ideas for Agents that, if I implement them, will help automate more tasks from my personal life.

For anyone else learning callin.io, I highly recommend finding something in your personal life that could be automated or something from your work, and try to automate it using callin.io. Your motivation will be much higher as you are trying to solve a personal pain point!

Comment here if anyone would like a more detailed, complete breakdown of my AI Agent.

I was thinking maybe a video going over everything could be useful?

FamilyMate.AI Roadmap

For now, my plan is to keep improving FamilyMate.AI and add new features and functionality.

• Telegram Text Support - Send and receive Telegram text messages
• Google Calendar Event Management - Manage the family shared calendar
• Voice Support - Talk to the AI Assistant, send voice messages instead of text
• AI Assistant Voice - Allow the AI Assistant to send voice messages in Telegram
• Image Support - Send an image, e.g., an image of a poster advertising some event
• Reminders - Telegram message with a reminder of upcoming events
• Grocery Shopping List Management
• TODO List Management

Which one of the new features would you like to see implemented next?

Can you think of any other tasks that FamilyMate could help with?

Happy Making,
Brian