Aymara Python SDK

Aymara Python SDK#

Hi! 👋 We’re Aymara, and we help teams at AI labs and Fortune 50 companies measure and improve the alignment of any generative AI model (e.g., a fine-tuned Llama) or application (e.g., a GPT-powered chatbot).

What You Can Do#

Use Aymara to create custom evals across safety, quality, and modalities:

Eval

What it measures

Status

🛡️ Safety

Content policy compliance and refusals

Available

🧨 Jailbreaks

Resilience to adversarial prompts

Available

🎯 Accuracy

Knowledge grounding and hallucination control

Available

🎬 Multimodal

Text-to-image and text-to-video

Available

🌐 Multilingual

EN, ES, FR, IT, PT, DE

Available

🗣️ Tone

Politeness, professionalism, de-escalation

Coming soon

🫡 Sycophancy

Undue agreement vs. truthful disagreement

Coming soon

🚨 Escalation

Proper refusal/hand-off for risky requests

Coming soon

🔄 Multiturn

Multi-step conversations and follow-ups

Coming soon

Aymara scores responses, explains failures in plain language, and surfaces actionable fixes to boost safety and accuracy.

You can see our tools in action in two recent research projects:

LLM Risk & Responsibility Matrix

Gender Bias in Text‑to‑Image

Key findings · arXiv report

Key findings · arXiv report

LLM Safety Benchmark matrix

Text-to-Image gender bias graph

Getting Started#

  1. Sign up and secure your API key Start with a free trial or request full access via a demo, then create an account to generate your key.

  2. Install the Python SDK

    pip install aymara-ai-sdk

    See the Python SDK README and SDK Reference for comprehensive usage details.

  3. Explore guided workflows
    Browse our User Guides to walk through real evaluation pipelines end to end.

Community & Support#

  • Track issues and feature requests on GitHub

  • Reach us at support@aymara.ai for help, feedback, or partnerships

  • Check the FAQs for quick troubleshooting tips

Contents:

Contents