LogoLogo
  • Introduction to the Ebbot Platform
  • Ebbot Platform
  • Bot basics
    • Scenarios
    • Entities
    • Triggers
    • Training center
  • Scenarios
    • Cards and syntax
      • File Input
      • Text card
      • Input
      • Buttons
      • Image
      • File
      • Carousel
      • Location
      • List
      • Contact Agent
      • Rating request
      • Custom component
      • CoBrowsing
    • Transition
    • Card properties
  • AI Insights
    • Setup and Configuration
    • Using the Insights Dashboard
  • EbbotGPT
    • Knowledge
      • Data source transformer
      • Source types
        • File
        • Website scrape
        • Docx file
        • TOPdesk API
        • Sitevision API
        • SharePoint API
          • Create app with Sites.FullControl.All permission in Azure
          • Ebbot SharePoint Postman Guide
        • Confluence API
    • Configurations
    • Persona
    • GPT Evaluation
    • Embedder models
    • EGPT models
  • Custom vocabulary
  • Tutorials
    • Create your first scenario
      • Select a trigger
      • Add bot responses
  • Data Object
  • Release notes
  • For developers
    • Ebbot SDK
    • Safe Exchange API / Vault
    • Subdomain manager
  • EbbotGPT API
  • Chatbot & Live chat
    • Install chat widget
    • Chats API
    • Chat widget API
    • Datasource API
    • Sales tracking for live chat
    • Webhook
      • Incoming webhooks
      • Outgoing webhooks
    • SMS API
      • Authentication
      • Send SMS
      • Errors
      • Encoding
    • Python components
    • Intent detection (NLP)
  • Product guides
    • Product data feeds
    • Install guide
    • Product guide events
      • Product guide user events
      • Received events
      • Send events
    • API & webhooks
    • GA4 integration
    • Klaviyo integration
  • Messenger marketing
    • Install popup
    • API & webhooks
  • For chat agents
    • Ebbot Chat
      • Settings modal
      • Queue
      • Topbar Stats
      • Menu
        • Power-Ups!
        • Quick Replies
  • INTEGRATIONS
    • Ebbot Live Chat in Zendesk
      • Setup guide
    • Active Directory - SAML
    • Configure SAML in Azure
Powered by GitBook
On this page
  • Evaluation set
  • Runs
  • Use cases

Was this helpful?

  1. EbbotGPT

GPT Evaluation

GPT evaluation allows you to test questions using a specific configuration to identify issues and compare results across different settings.

PreviousPersonaNextEmbedder models

Last updated 9 days ago

Was this helpful?

Evaluation set

An evaluation set is a collection of questions you want to test. You can add questions manually or import them from a CSV file.

Question: The message you want to test with the LLM.

Expected answer: The answer you're expecting from the LLM. This field is optional and can be left empty.

The expected answer is currently not automatically compared to the actual output. Use it as a reference to remind yourself of what answer you expect from the model.

Runs

A run is the process of testing questions against the LLM using your defined settings. You can compare different runs to see how changes, such as using a different GPT model or persona, affect the answers.

Runs may take some time to complete, depending on the time of day they are started. They are processed with lower priority and placed in a queue to avoid impacting the speed and performance of the LLMs.

Export a Run

You can export a run as a CSV from the Settings view or from the Run view. In the settings view you can select to only support export selected questions or all questions.

Use cases

Testing a new LLM

You want to switch to a newly released GPT model and ensure your existing configuration settings still work as expected.

Changing the persona

You’ve just made changes to the persona to solve a specific issue, and now you want to ensure those changes haven’t negatively affected the output for other questions.

New sources

You’ve added or updated sources and want to see how those changes affect the model’s output.