Lunary ✓

Lunary is an open-source AI developer platform to monitor, manage, and improve LLM apps. It performs essential functions like logs and debugging of LLM agents, enabling developers to trace errors and label data for fine-tuning.

Lunary also provides a cost monitoring feature, segmented by user and model; this aids in optimizing costs. With its benchmarking feature, you can experiment with prompts and models to find the best performing ones.

Lunary enables recording of user conversations, helping identify gaps in your chatbot's knowledge. It also has a provision to help you create templates and collaborate on prompts with non-technical teammates.

Other notable features include hosting for your data with secure self-host options and compliances to ensure maximum security. Furthermore, it provides an open-source community for developers, live tail features, an efficient search function, label data, prompts and a complete API.

Lunary also provides alerts for outlier results and errors.