skip to main content
10.1145/3639856.3639921acmotherconferencesArticle/Chapter ViewAbstractPublication PagesaimlsystemsConference Proceedingsconference-collections
demonstration

Democratic Evaluation Framework among Organizations

Published: 17 May 2024 Publication History

Abstract

This paper presents a democratic evaluation framework designed for different teams within an organization to assess the latest groundbreaking advancements in Generative AI for text. The framework harmoniously integrates efforts from AI developers, who possess expertise in AI and are actively providing AI solutions, with users seeking to leverage AI components in their products, fostering an inclusive AI community. By encouraging collaboration among developer teams in evaluating Generative AI, the framework accelerates solution development, reducing time and costs. Moreover, the proposed evaluation framework offers flexibility in including and expanding upon unknown components in generative AI space use cases. It has undergone testing on state-of-the-art large language models and is employed to evaluate different components in the organization, promoting cross-team collaboration and achieving significant reductions in time and costs.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
AIMLSystems '23: Proceedings of the Third International Conference on AI-ML Systems
October 2023
381 pages
ISBN:9798400716492
DOI:10.1145/3639856
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 May 2024

Check for updates

Author Tags

  1. Automated Evaluation
  2. Generative AI
  3. Large Language Models
  4. Natural Language Generation

Qualifiers

  • Demonstration
  • Research
  • Refereed limited

Conference

AIMLSystems 2023

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 11
    Total Downloads
  • Downloads (Last 12 months)11
  • Downloads (Last 6 weeks)3
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media