Workspaces

Workspaces#

Workspaces help you organize multiple teams in your company that are using Aymara to evaluate their AI models or applications. They provide a clear way to group users, manage visibility, and track billing by team.

A Workspace is a collection of team members where:

  • Evaluation outputs (like evals, score runs, and reports) are automatically associated with that workspace.

  • All output is viewable by every member of the workspace by default, making team collaboration seamless.

  • Billing for all activities is consolidated for the entire workspace.

Key things to know:

  • A user must belong to a workspace to create evals or perform other billable actions.

  • Each user can only belong to one workspace at a time.

Managing Workspaces (For Admins)#

Organization admins can manage everything from the Workspaces tab in the Aymara app. From here, you can:

  • Create, edit, and delete workspaces.

  • Add users to or remove them from a workspace.

If a user isn’t assigned to a workspace, they won’t be able to run evaluations and will be prompted to contact an admin for access.

Workspaces management UI in the Aymara app

SDK Usage#

When you create objects like evals, score runs, or reports using the SDK, they are automatically associated with your user’s workspace.

Accessing Objects#

By default, list methods will only return objects from your own workspace.

To access data from other workspaces, use the workspace_uuid parameter in methods like list(), list_prompts(), list_responses(), etc.

  • Access a specific workspace: Pass its unique ID.

    # Example: List evals from another team's workspace
aymara_client.evals.list(workspace_uuid="workspace.a1b2c3d4-e5f6-...")

  • Access all workspaces in the organization: Use the wildcard '*'.

    # Example: List all reports across the entire organization
aymara_client.reports.list(workspace_uuid='*')

If you attempt to create an object without being a member of a workspace, the API will raise a PermissionDeniedError. The error includes the emails of your organization’s admins, who can add you to your relevant workspace.

from aymara_ai import AymaraAI

aymara_client = AymaraAI()

aymara_client.evals.create(
    ai_description="The AI is a chatbot that responds to queries from Gotham citizens.",
    ai_instructions="Do not reveal Wayne Enterprises' trade secrets.",
    eval_type="safety",
)

Traceback (most recent call last):
  ...
aymara.errors.PermissionDeniedError: Error code: 403 - {'error': {'code': 'auth.workspace_required', 'message': 'Workspace assignment required', 'details': {'admin_contacts': [{'email': 'lucius@wayne.com'}, {'email': 'bruce@wayne.com'}]}}, 'request_id': 'c7208424-af4b-4225-b945-4118eadfa315'}

Billing & Usage#

Admins can navigate to the Billing & Usage page to view data for the entire organization. The view can be filtered by one or more workspaces, making it easy to understand usage and costs per team.

Billing and Usage dashboard filtered by workspace

If you find a bug, have a question, or want to request a feature, say hello at support@aymara.ai or open an issue on our GitHub repo.

Contents