AskYourDatabase in AI Database Agents
Scenario-level performance from current published Results.
24 of 28 scenarios published
Where AskYourDatabase failed
7 of 24 graded scenarios. Open a Result to see the conversation and the evidence.
The operator needs to find the questions the agent is failing on
AskYourDatabase did not surface unanswered questions as failures. It placed two questions it could not answer in the same Messages list style as three it answered.
View Result →The answer is a trend or a comparison
AskYourDatabase returned monthly order counts without volunteering a visual to show the trend. Its answer used a text table and described the increase in words.
View Result →The agent offers what to ask next
AskYourDatabase did not suggest a follow-up question after answering the query. The answer ended the thread with no suggestion chips or follow-up list, and the only suggestion-like setting is for new chat sessions.
View Result →The follow-up is ambiguous
AskYourDatabase did not ask for clarification when a follow-up was ambiguous. The first answer created two distinct customer lists, and the second turn answered with a side-by-side comparison instead of asking which list was meant.
View Result →An excluded table is needed to answer the question
AskYourDatabase fails this scenario: it answers from another table instead of saying it cannot answer when the hidden table is excluded. The access-control screen shows the hidden table set to HIDE while the other tables stay readable, and the chat shows a count from a support-ticket queue rather than a refusal.
View Result →The report is run again after the data has changed
AskYourDatabase answered the relative-period order question, but no save control was found for rerunning the answer as a report after data changed. The answer actions offered copying and PDF printing.
View Result →The user wants to keep an answer they just got
No save control was found in AskYourDatabase for keeping the answer and reopening it in a new session. Its bot settings, data source, billing, API key and settings pages offered no save control, and creating a dashboard page returned an error.
View Result →How AskYourDatabase performed
Open a capability to explore its scenarios. Each row reports the test set in its published Result.
Analytics & Observability2 pass1 fail1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| Reported analytics match what actually happened | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase's reported counts matched what was run in the two accounts. The first account showed 3 conversations and the second showed 2, and the messages list showed the five questions with timestamps; each count included an unanswered chat. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| Someone needs to see what people have been asking | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase showed what people had asked and which account asked each question. Its usage list contained five entries for the five questions asked. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The operator needs to find the questions the agent is failing on | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedAskYourDatabase did not surface unanswered questions as failures. It placed two questions it could not answer in the same Messages list style as three it answered. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| A user reports a bad answer and the operator has to find it | No published result | ||
Answer Presentation3 pass1 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The answer is a plain fact or a short list | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase met this presentation requirement in the tested case. It stated 308 total orders in a plain sentence, and the screen showed no chart, plot, result grid, or one-cell table. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The answer is a trend or a comparison | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedAskYourDatabase returned monthly order counts without volunteering a visual to show the trend. Its answer used a text table and described the increase in words. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The answer needs a summary and its detail together | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase gave a headline figure and a breakdown in forms suited to each, so it met the scenario. It returned a revenue summary table for the total, a category table for the breakdown, and takeaways naming the driver; there was no chart. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The user asks to see the same answer a different way | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase presented the same category revenue figures as a horizontal bar chart on request. All seven amounts stayed the same to the cent, including outdoor and patio revenue of $7,744.59 in both forms. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Conversational Interaction4 pass2 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| Customer corrects information given earlier | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase answered the corrected period only. The July question returned 23 orders, and the follow-up June correction produced a new June query and 10 orders, with the thread ending there. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The agent offers what to ask next | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedAskYourDatabase did not suggest a follow-up question after answering the query. The answer ended the thread with no suggestion chips or follow-up list, and the only suggestion-like setting is for new chat sessions. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The follow-up is ambiguous | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedAskYourDatabase did not ask for clarification when a follow-up was ambiguous. The first answer created two distinct customer lists, and the second turn answered with a side-by-side comparison instead of asking which list was meant. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The next question refers to the previous answer | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase kept the follow-up tied to the previous answer. Turn 1 returned five customers, and turn 2 used the same five without a clarifying question or a silent change in the set. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The user changes direction mid-conversation | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase handles the mid-conversation change by dropping the earlier Texas filter and answering with the broader all-states breakdown. The second query has no state condition, and the 15-row table totals 308 orders and 67 cancellations; the Texas turn stays separate at 18 orders and 27.78%. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The user narrows what they just asked | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase handles this narrowing case: it keeps the earlier question’s date window and adds the new Texas constraint. In the first turn, the app returned 59 orders in a 10-row preview plus a full CSV. In the follow-up, it kept the same window, returned 3 Texas orders, and those three all appear in the first turn’s 59-row CSV. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Data Access Control2 pass1 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| An excluded column is asked for directly | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase reported the directly requested API token as unavailable without revealing its value. Its executed query did not read the token, while other customer details remained available. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| An excluded table is needed to answer the question | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedAskYourDatabase fails this scenario: it answers from another table instead of saying it cannot answer when the hidden table is excluded. The access-control screen shows the hidden table set to HIDE while the other tables stay readable, and the chat shows a count from a support-ticket queue rather than a refusal. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| Two users are given different data scope | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase kept the two answers within their separate scopes. The full-access account gave Alice Prosser's plus tier and active status, while the restricted account gave no tier and no Alice-specific value. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Question Answering3 pass1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The connected data cannot answer the question | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase could not answer the question from the connected data. It said the schema it could see had no review or rating data, asked where that data is stored, and returned no number. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The question uses a term the company defines itself | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase stated its chosen definition of revenue rather than treating the company-defined term as unambiguous. It returned $24,944.28 based on order totals with cancelled orders excluded. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The user's wording doesn't match how the values are stored | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase returned 14 gift-card orders for a request about gift vouchers. The table matched the reference answer's $16,806.08 total and status breakdown. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The answer is available in the connected data | No published result | ||
Reporting1 pass2 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The report is run again after the data has changed | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedAskYourDatabase answered the relative-period order question, but no save control was found for rerunning the answer as a report after data changed. The answer actions offered copying and PDF printing. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The user wants several findings in one place | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase brought several findings together in one response. Its report combined revenue, sales category and cancellation figures that agreed with its direct answers to the same questions. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The user wants to keep an answer they just got | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedNo save control was found in AskYourDatabase for keeping the answer and reopening it in a new session. Its bot settings, data source, billing, API key and settings pages offered no save control, and creating a dashboard page returned an error. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Training2 pass2 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The database's names are cryptic and the user documents them | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase used buyer-written documentation to reach the requested data. It counted 7 returns flagged for manual fraud review, matching the reference count of 7. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The user sets a standing rule for all future answers | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase passed this scenario: a standing rule carried into later, unrelated answers. The exclusion was entered in Documentation once, and both later replies used it in the generated query and in the answer text. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The agent used the wrong definition and the user corrects it | No published result | ||
| The user supplies example questions and the queries they trust | No published result | ||
Reading these Results
Published evidence and test coverage answer different questions.
Which scenarios have a Result?
A published Result is public evidence for this tool on one scenario. “No published result” does not say whether testing has taken place.
What does each Result cover?
Assessed includes Pass, Fail and Not gradable. Gradable includes Pass and Fail. Both use the pinned test count in that published Result.
Out of 28 scenarios
The summary counts each of the 28 scenarios once. A scenario fails when any of its published test cases failed. “No published Result yet” means nothing is public for that scenario; it does not say whether AskYourDatabase was tested on it.