Overview

A research project at a premier Business School required analysis of news articles on scale.

In today’s time LLM is the right tool for such tasks. On surface level, a simple prompt can give an analysis and determine the theme of the news. 

However, it becomes challenging with changing needs, specialized categories, making sure LLM output is free of any kind of bias, and error free.

The problem

• The team needed to iterate on prompts with changing requirements.
  
  

• Professors and Research associates (without coding backgrounds) needed to quickly test their prompts, and make changes in the prompts written by developers.
  
  

• They needed to test their prompts multiple times on smaller datasets before handing over for a technical implementation.

This would naturally slow down the process. A research associate would write a prompt in ChatGPT, test it with a couple of news articles, and then share it through an email thread (usually based on gut feeling).

Another teammate will suggest a change (or there is a feedback from a professor), and you have two new emails — and we've not even tested it yet.

This will then be tested by someone in the tech team on a CSV file, sent back to the associates & professor for review, adding to the delay. And also inbox.

The solution

Before anything, the team needed a platform where both technical and domain experts could collaborate. 

With Flapico, people within an organization write, share, and version prompts. The team created a dedicated workspace for this project, and were able to iterate and finalize prompts way quicker. 

The domain experts were able to upload smaller datasets, and test their prompts before making a decision. The quantitative testing gives them the confidence & numbers before the tech team integrates their prompts in their pipeline.

Flapico also becomes the single source of truth for the final version (often consumed through APIs).

Results

• About 70% faster iteration cycles

• Lesser emails

• Zero misalignment between non-tech and tech stakeholders

• Easier handoff of insights to future collaborators or students