Create Your First Project

Start adding your projects to your portfolio. Click on "Manage Projects" to get started

Product design for analyzing AI-generated text datasets

Project type

Data Visualisation, Ideation

Date

April 2024

Location

London

Map out a workflow and create a user interface to inspect unwieldy datasets of AI-generated text. Find various methods to surface data for analyzing and adjusting the behaviour of the LLM model, including identifying patterns, errors, and areas for improvement.

Key criteria and metrics for prompt and logic adjustment

GOAL:
Explore diverse methods to present data effectively, taking into account its type and dynamic nature. Utilize a range of visualization techniques to highlight clusters within datasets, enabling to delve into specific segments for detailed analysis.

DELIVERABLES:
- Map out workflow levels, data categorization, and metric types.
- Create low-fidelity wireframes for each metric type with interaction logic at different depth levels

We started the project by defining key terminology, criteria and metrics to assess the effectiveness of AI prompt adjustment.

We organized these metrics based on their type, definition, and context of use, already gaining insights into the key decisions they inform, their potential range, and the specific patterns or outcomes we expect.

Acknowledging the model's analysis will evolve, we understood there might be patterns we haven't discovered yet.

Workflow mapping - decision-making process for prompt adjustment

COLLABORATION TYPE:
- Highly interactive collaborative with product owner/developer, brainstorming sessions, collaborative whiteboard designs

Workflow mapping for prompt adjustment aims to establish a systematic approach to efficiently condense extensive content (e.g. a book) into concise factual summaries (e.g. Wikipedia article).

Page structure to integrate analysis across various metric levels

During mapping, we identified 3 levels of metrics analysis:
- collection
- requests
- choices

Data visualization ideation specific to individual metric types

We ideated possible charts, and data tables to visualise each metric type and model response distribution across three levels.

Metrics:
- Split location
- Compression ratio
- Split topic
- Language
- Token count
- Error
- Quality etc

Creating wireframes and interactions for each level of metric analysis. Collection level

Here is an example of the compression ratio distribution on the collection level, and we have also created wireframes for other metrics such as split ratio, language, errors, and quality score

Creating wireframes and interactions for each level of metric analysis. Request level

Here is an example of the compression ratio distribution on the request level, and we have also created wireframes for other metrics such as split ratio, language, errors, and quality score

Creating wireframes and interactions for each level of metric analysis. Choice level

Example of each choice that includes AI-generated summary and comparing them to a baseline

.pro-gallery-wix-wrapper {display: block !important;} .pro-gallery-wix-wrapper .gallery-item-container {opacity: 1 !important; display: block !important;}

Polina Yakymenko
Expressive Arts & Design

Create Your First Project

Product design for analyzing AI-generated text datasets

Project type

Data Visualisation, Ideation

Date

April 2024

Location

London

Map out a workflow and create a user interface to inspect unwieldy datasets of AI-generated text. Find various methods to surface data for analyzing and adjusting the behaviour of the LLM model, including identifying patterns, errors, and areas for improvement.

Polina Yakymenko

Expressive Arts Facilitatorin
Designerin & Forscherin

Eine Nachricht senden

Vielen Dank für Ihre Kontaktaufnahme! Ich werde Ihre E-Mail innerhalb von 1-2 Werktagen beantworten

Polina Yakymenko Expressive Arts & Design

Create Your First Project

Product design for analyzing AI-generated text datasets

Project type

Data Visualisation, Ideation

Date

April 2024

Location

London

Map out a workflow and create a user interface to inspect unwieldy datasets of AI-generated text. Find various methods to surface data for analyzing and adjusting the behaviour of the LLM model, including identifying patterns, errors, and areas for improvement.

Polina Yakymenko

Expressive Arts Facilitatorin Designerin & Forscherin

Eine Nachricht senden

Vielen Dank für Ihre Kontaktaufnahme! Ich werde Ihre E-Mail innerhalb von 1-2 Werktagen beantworten

Polina Yakymenko
Expressive Arts & Design

Expressive Arts Facilitatorin
Designerin & Forscherin