Tool in benchmark · Version 1

AskYourDatabase in AI Database Agents

Scenario-level performance from current published Results.

17Passed
7Failed
4No published Result yet

24 of 28 scenarios published

CapabilityScenariosResult
Analytics & Observability2 pass · 1 fail · 3 of 4 published
Answer Presentation3 pass · 1 fail · 4 of 4 published
Conversational Interaction4 pass · 2 fail · 6 of 6 published
Data Access Control2 pass · 1 fail · 3 of 3 published
Question Answering3 pass · 0 fail · 3 of 4 published
Reporting1 pass · 2 fail · 3 of 3 published
Training2 pass · 0 fail · 2 of 4 published
PassedFailedPublished, different test setNo published Result yet

Where AskYourDatabase failed

7 of 24 graded scenarios. Open a Result to see the conversation and the evidence.

Analytics & Observability

The operator needs to find the questions the agent is failing on

AskYourDatabase did not surface unanswered questions as failures. It placed two questions it could not answer in the same Messages list style as three it answered.

View Result →
Answer Presentation

The answer is a trend or a comparison

AskYourDatabase returned monthly order counts without volunteering a visual to show the trend. Its answer used a text table and described the increase in words.

View Result →
Conversational Interaction

The agent offers what to ask next

AskYourDatabase did not suggest a follow-up question after answering the query. The answer ended the thread with no suggestion chips or follow-up list, and the only suggestion-like setting is for new chat sessions.

View Result →
Conversational Interaction

The follow-up is ambiguous

AskYourDatabase did not ask for clarification when a follow-up was ambiguous. The first answer created two distinct customer lists, and the second turn answered with a side-by-side comparison instead of asking which list was meant.

View Result →
Data Access Control

An excluded table is needed to answer the question

AskYourDatabase fails this scenario: it answers from another table instead of saying it cannot answer when the hidden table is excluded. The access-control screen shows the hidden table set to HIDE while the other tables stay readable, and the chat shows a count from a support-ticket queue rather than a refusal.

View Result →
Reporting

The report is run again after the data has changed

AskYourDatabase answered the relative-period order question, but no save control was found for rerunning the answer as a report after data changed. The answer actions offered copying and PDF printing.

View Result →
Reporting

The user wants to keep an answer they just got

No save control was found in AskYourDatabase for keeping the answer and reopening it in a new session. Its bot settings, data source, billing, API key and settings pages offered no save control, and creating a dashboard page returned an error.

View Result →

How AskYourDatabase performed

Open a capability to explore its scenarios. Each row reports the test set in its published Result.

Analytics & Observability2 pass1 fail1 no Result yet
ScenarioPublished outcomesTest coverageResult
Reported analytics match what actually happened
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase's reported counts matched what was run in the two accounts. The first account showed 3 conversations and the second showed 2, and the messages list showed the five questions with timestamps; each count included an unanswered chat.

1/1 assessed1/1 gradablePublished test setView Result →
Someone needs to see what people have been asking
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase showed what people had asked and which account asked each question. Its usage list contained five entries for the five questions asked.

1/1 assessed1/1 gradablePublished test setView Result →
The operator needs to find the questions the agent is failing on
0 Pass1 Fail0 Not gradable
1 of 1 test case failed
What happened

AskYourDatabase did not surface unanswered questions as failures. It placed two questions it could not answer in the same Messages list style as three it answered.

1/1 assessed1/1 gradablePublished test setView Result →
A user reports a bad answer and the operator has to find itNo published result
Answer Presentation3 pass1 fail
ScenarioPublished outcomesTest coverageResult
The answer is a plain fact or a short list
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase met this presentation requirement in the tested case. It stated 308 total orders in a plain sentence, and the screen showed no chart, plot, result grid, or one-cell table.

1/1 assessed1/1 gradablePublished test setView Result →
The answer is a trend or a comparison
0 Pass1 Fail0 Not gradable
1 of 1 test case failed
What happened

AskYourDatabase returned monthly order counts without volunteering a visual to show the trend. Its answer used a text table and described the increase in words.

1/1 assessed1/1 gradablePublished test setView Result →
The answer needs a summary and its detail together
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase gave a headline figure and a breakdown in forms suited to each, so it met the scenario. It returned a revenue summary table for the total, a category table for the breakdown, and takeaways naming the driver; there was no chart.

1/1 assessed1/1 gradablePublished test setView Result →
The user asks to see the same answer a different way
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase presented the same category revenue figures as a horizontal bar chart on request. All seven amounts stayed the same to the cent, including outdoor and patio revenue of $7,744.59 in both forms.

1/1 assessed1/1 gradablePublished test setView Result →
Conversational Interaction4 pass2 fail
ScenarioPublished outcomesTest coverageResult
Customer corrects information given earlier
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase answered the corrected period only. The July question returned 23 orders, and the follow-up June correction produced a new June query and 10 orders, with the thread ending there.

1/1 assessed1/1 gradablePublished test setView Result →
The agent offers what to ask next
0 Pass1 Fail0 Not gradable
1 of 1 test case failed
What happened

AskYourDatabase did not suggest a follow-up question after answering the query. The answer ended the thread with no suggestion chips or follow-up list, and the only suggestion-like setting is for new chat sessions.

1/1 assessed1/1 gradablePublished test setView Result →
The follow-up is ambiguous
0 Pass1 Fail0 Not gradable
1 of 1 test case failed
What happened

AskYourDatabase did not ask for clarification when a follow-up was ambiguous. The first answer created two distinct customer lists, and the second turn answered with a side-by-side comparison instead of asking which list was meant.

1/1 assessed1/1 gradablePublished test setView Result →
The next question refers to the previous answer
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase kept the follow-up tied to the previous answer. Turn 1 returned five customers, and turn 2 used the same five without a clarifying question or a silent change in the set.

1/1 assessed1/1 gradablePublished test setView Result →
The user changes direction mid-conversation
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase handles the mid-conversation change by dropping the earlier Texas filter and answering with the broader all-states breakdown. The second query has no state condition, and the 15-row table totals 308 orders and 67 cancellations; the Texas turn stays separate at 18 orders and 27.78%.

1/1 assessed1/1 gradablePublished test setView Result →
The user narrows what they just asked
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase handles this narrowing case: it keeps the earlier question’s date window and adds the new Texas constraint. In the first turn, the app returned 59 orders in a 10-row preview plus a full CSV. In the follow-up, it kept the same window, returned 3 Texas orders, and those three all appear in the first turn’s 59-row CSV.

1/1 assessed1/1 gradablePublished test setView Result →
Data Access Control2 pass1 fail
ScenarioPublished outcomesTest coverageResult
An excluded column is asked for directly
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase reported the directly requested API token as unavailable without revealing its value. Its executed query did not read the token, while other customer details remained available.

1/1 assessed1/1 gradablePublished test setView Result →
An excluded table is needed to answer the question
0 Pass1 Fail0 Not gradable
1 of 1 test case failed
What happened

AskYourDatabase fails this scenario: it answers from another table instead of saying it cannot answer when the hidden table is excluded. The access-control screen shows the hidden table set to HIDE while the other tables stay readable, and the chat shows a count from a support-ticket queue rather than a refusal.

1/1 assessed1/1 gradablePublished test setView Result →
Two users are given different data scope
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase kept the two answers within their separate scopes. The full-access account gave Alice Prosser's plus tier and active status, while the restricted account gave no tier and no Alice-specific value.

1/1 assessed1/1 gradablePublished test setView Result →
Question Answering3 pass1 no Result yet
ScenarioPublished outcomesTest coverageResult
The connected data cannot answer the question
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase could not answer the question from the connected data. It said the schema it could see had no review or rating data, asked where that data is stored, and returned no number.

1/1 assessed1/1 gradablePublished test setView Result →
The question uses a term the company defines itself
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase stated its chosen definition of revenue rather than treating the company-defined term as unambiguous. It returned $24,944.28 based on order totals with cancelled orders excluded.

1/1 assessed1/1 gradablePublished test setView Result →
The user's wording doesn't match how the values are stored
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase returned 14 gift-card orders for a request about gift vouchers. The table matched the reference answer's $16,806.08 total and status breakdown.

1/1 assessed1/1 gradablePublished test setView Result →
The answer is available in the connected dataNo published result
Reporting1 pass2 fail
ScenarioPublished outcomesTest coverageResult
The report is run again after the data has changed
0 Pass1 Fail0 Not gradable
1 of 1 test case failed
What happened

AskYourDatabase answered the relative-period order question, but no save control was found for rerunning the answer as a report after data changed. The answer actions offered copying and PDF printing.

1/1 assessed1/1 gradablePublished test setView Result →
The user wants several findings in one place
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase brought several findings together in one response. Its report combined revenue, sales category and cancellation figures that agreed with its direct answers to the same questions.

1/1 assessed1/1 gradablePublished test setView Result →
The user wants to keep an answer they just got
0 Pass1 Fail0 Not gradable
1 of 1 test case failed
What happened

No save control was found in AskYourDatabase for keeping the answer and reopening it in a new session. Its bot settings, data source, billing, API key and settings pages offered no save control, and creating a dashboard page returned an error.

1/1 assessed1/1 gradablePublished test setView Result →
Training2 pass2 no Result yet
ScenarioPublished outcomesTest coverageResult
The database's names are cryptic and the user documents them
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase used buyer-written documentation to reach the requested data. It counted 7 returns flagged for manual fraud review, matching the reference count of 7.

1/1 assessed1/1 gradablePublished test setView Result →
The user sets a standing rule for all future answers
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase passed this scenario: a standing rule carried into later, unrelated answers. The exclusion was entered in Documentation once, and both later replies used it in the generated query and in the answer text.

1/1 assessed1/1 gradablePublished test setView Result →
The agent used the wrong definition and the user corrects itNo published result
The user supplies example questions and the queries they trustNo published result

Reading these Results

Published evidence and test coverage answer different questions.

Publication availability

Which scenarios have a Result?

A published Result is public evidence for this tool on one scenario. “No published result” does not say whether testing has taken place.

Test coverage

What does each Result cover?

Assessed includes Pass, Fail and Not gradable. Gradable includes Pass and Fail. Both use the pinned test count in that published Result.

Scenario scope

Out of 28 scenarios

The summary counts each of the 28 scenarios once. A scenario fails when any of its published test cases failed. “No published Result yet” means nothing is public for that scenario; it does not say whether AskYourDatabase was tested on it.

AskYourDatabase in AI Database Agents | AI Demos