Eval Datasets

4️⃣2️⃣ Evaluation Datasets

Mission 🚀

To showcase 42 Eval datasets for a range of different industries and types of evaluation for beginners, intermediates as well as experts.

H2O.ai LLM Evaluation Datasets

The evaluation datasets in this collection are for a range of different evaluation types:

Conditional Questions : The complexity of the question is increased by incorporating a scenario or condition that affects the context of the question.
Multi-hop Reasoning Questions : The complexity of the question is increased, where the question should require the reader to make multiple logical connections or inferences using the information available in given context.
Token Presence : Extract the minimum tokens necessary for accurate answers for the question.

The datasets in this project are generated based on documents for different industries, for example:

Banking
Risk Management
Telecom
Government
Legal
Health
Finance
Security

Available Datasets

Following is the collection of 40 Datasets by different industries, different evaluation types, built in H2O LLM Data Studio. Click on any dataset to get more details.

#	Dataset Name	Industry	Sub Industry	No of Entries	Prompt Type
1.	Banking_Financial_Statements_Alphabet_Tesla	`Banking`	Company financial statement	520	RAG
2.	Risk_Management_Policy	`Risk Management`	Risk Management	160	RAG
3.	Telcom_Technical_Report	`Telecom`	Technical Report	130	RAG
4.	Government_Constitution_of_the_Republic_of_Singapore	`Government`	Government	160	RAG
5.	Healthcare_Health_Service_Standards	`Health`	Health Service Standards	82	RAG
6.	Legal_EU_AI_Act	`Legal`	EU AI Act	143	RAG
7.	Legal_IRS_Document_1	`Legal`	IRS Filing Policy	145	RAG
8.	Legal_IRS_Document_2	`Legal`	IRS Filing Policy	126	RAG
9.	Banking_CBA_AnnualReport_2023	`Banking`	Annual Report	120	RAG
10.	Government_Home_Affairs	`Government`	Home Affairs	116	RAG
11.	Finance_Product_Disclosure_Statement	`Finance`	Product Disclosure Statement	123	RAG
12.	Telcom_Telecommunication_Regulations	`Telcom`	Telecommunications Regulations	114	RAG
13.	Banking_Brokers_Agreement	`Banking`	Brokers Agreement	58	RAG
14.	Security_Singapore_Cyber_Landscape	`Security`	Singapore Cyber Landscape	104	RAG
15.	Risk_Management_Inherent_Risk_Assessment	`Risk Management`	Inherent Risk Assessment	116	RAG
16.	Government_Singapore_Labour_Force	`Government`	Singapore Labour Force	113	RAG
17.	Banking_HSBC_Annual_Report	`Banking`	Annual Report	121	RAG
18.	Government_US_Veterans_Affairs	`Government`	Veterans Affairs	121	RAG
19.	Security_Cyber_Security_Policy	`Security`	Cyber Security	125	RAG
20.	Risk_Management_Risk_Management_Guidelines	`Risk Management`	Risk Management Guidelines	123	RAG
21.	Telcom_Annual_Report_Singtel	`Telcom`	Annual Report	153	RAG
22.	Legal_IRS_Strategic_Operating_Plan	`Legal`	IRS Strategic Operating Plan	128	RAG
23.	Government_Immigration_in_Singapore	`Government`	Immigration in Singapore	122	RAG
24.	Telcom_Telecommunications_regulation_strategy_policy	`Telecom`	Telecommunications Regulations	116	RAG
25.	Healthcare_Stanford_Healthcare_Regulations	`Health`	Health Regulations	114	RAG
26.	Banking_Policy_Document	`Banking`	Policy Document	125	RAG
27.	Finance_Financial_Records_Management	`Finance`	Financial Document	135	RAG
28.	Security_Information_Security_Policy	`Security`	Information Security Policy	146	RAG
29.	Government_Defense_Management	`Government`	Defense Management	137	RAG
30.	Legal_Employment_Contract	`Legal`	Employment Contract	60	RAG
31.	Healthcare_Digital_Health_Guidelines	`Health`	Digital Health Guidelines	127	RAG
32.	Risk_Management_Risk_Management_Techniques_Tool	`Risk Management`	Risk Management	120	RAG
33.	Telcom_Telecom_Infrastructure_Planning	`Telcom`	Telecom Infrastructure Planning	153	RAG
34.	Banking_Banking_Act	`Banking`	Banking Act	148	RAG
35.	Finance_IMF_Financial_Statements	`Finance`	Financial Statement	126	RAG
36.	Security_Information_Security	`Security`	Information Security	125	RAG
37.	Government_SA_Home_Affairs	`Government`	Home Affairs	140	RAG
38.	Healthcare_Maternal_Child_Healthcare	`Health`	Maternal Child Healthcare	147	RAG
39.	Legal_UPC_Agreement	`Legal`	UPC Agreement	141	RAG
40.	HR_HR_Policy	`HR`	HR Policy	122	RAG
41.	Telcom_Telcom_Customer_Service_Information	`Telcom`	Telcom Customer Service Information	133	RAG
42.	HR_HR_Policy_Procedures	`HR`	HR Policy	148	RAG

Create your own Evaluation Dataset using LLM DataStudio 💡

If you're interested in creating your own evaluation dataset using LLM DataStudio, you can follow the step-by-step guide provided in our dataset creation guide.

Using the Dataset in EvalStudio 📊

Bring-your-own Evals from LLM DataStudio seamlessly integrates in H2O EvalStudio. To utilize these datasets in EvalStudio for your own evaluations, follow the steps in the evaluation creation guide

Contribution 🛠️

Please create an Issue for any improvements, suggestions or errors in the content.

h2o-evals

README