Edcafe AI in Quiz Generation
Scenario-level performance from current published Results.
7 of 16 scenarios published
Where Edcafe AI failed
4 of 7 graded scenarios. Open a Result to see the conversation and the evidence.
A teacher gives the quiz to a class
Edcafe AI did not fully meet the requirements for giving a quiz to a class. Three learners completed the quiz through Google Classroom and had separate submissions, but the report did not identify their class or Google accounts.
View Result →A cleanly scanned document
Edcafe AI fails to turn a cleanly scanned document into quiz questions. The scanned PDF was attached in the Files tab, but Generate Quiz returned a text-extraction error naming that file and no quiz appeared.
View Result →Important information appears in both text and a diagram
Edcafe AI failed this scenario. It generated a complete 30-card quiz, but the diagram's unique feeding relationships were not tested, and three marked answers conflicted with the quiz's own arrow rule. The text ideas were covered, but the diagram-specific core idea was missed.
View Result →The user turns study material into a quiz
Edcafe AI turns study material into a quiz, but the Hindi run fails on language fidelity. The English run produced ten multiple-choice cards with correct marked answers and full chapter coverage; the Hindi run also produced ten cards, but with mangled chapter terms and English loanwords.
View Result →How Edcafe AI performed
Open a capability to explore its scenarios. Each row reports the test set in its published Result.
Answer Feedback1 pass1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The learner gives a clearly wrong answer | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedEdcafe AI passed this scenario. The learner-facing feedback said the answer was incorrect, gave the rule that current stays the same in a series circuit, and addressed the stated misunderstanding rather than only showing the answer. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The learner's answer is partly correct | No published result | ||
Grading1 pass2 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The learner answers correctly in their own words | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedEdcafe AI passes this own-words meaning-grading case. It marked Learner D Excellent Understanding Demonstrated, and the report showed a 0% average across four submissions while three learners were labelled excellent, so the average does not reflect this answer's wording. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The learner answers some fixed-answer questions correctly and others incorrectly | No published result | ||
| The learner's answer is partly correct | No published result | ||
Quiz Delivery1 pass1 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A learner takes a quiz on their own | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedEdcafe AI passes this scenario. One learner completed the quiz, submitted once, and the teacher report saved that attempt. Before submission, the quiz showed no answer information; after submission, the result screen showed the score and review. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| A teacher gives the quiz to a class | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedEdcafe AI did not fully meet the requirements for giving a quiz to a class. Three learners completed the quiz through Google Classroom and had separate submissions, but the report did not identify their class or Google accounts. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Quiz Generation3 fail2 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A cleanly scanned document | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedEdcafe AI fails to turn a cleanly scanned document into quiz questions. The scanned PDF was attached in the Files tab, but Generate Quiz returned a text-extraction error naming that file and no quiz appeared. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| Important information appears in both text and a diagram | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedEdcafe AI failed this scenario. It generated a complete 30-card quiz, but the diagram's unique feeding relationships were not tested, and three marked answers conflicted with the quiz's own arrow rule. The text ideas were covered, but the diagram-specific core idea was missed. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The user turns study material into a quiz | 1 Pass1 Fail0 Not gradable 1 of 2 test cases passed · 1 failedWhat happenedEdcafe AI turns study material into a quiz, but the Hindi run fails on language fidelity. The English run produced ten multiple-choice cards with correct marked answers and full chapter coverage; the Hindi run also produced ten cards, but with mangled chapter terms and English loanwords. | 2/2 assessed2/2 gradablePublished test set | View Result → |
| The user asks the quiz to test a particular learning goal | No published result | ||
| The user turns a recorded lecture into a quiz | No published result | ||
Class Reports1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The teacher checks what the class struggled with | No published result | ||
Learner Progress1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The learner checks whether they are improving | No published result | ||
Personalized Practice2 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| One topic is weaker than the others | No published result | ||
| The learner returns to the material later | No published result | ||
Reading these Results
Published evidence and test coverage answer different questions.
Which scenarios have a Result?
A published Result is public evidence for this tool on one scenario. “No published result” does not say whether testing has taken place.
What does each Result cover?
Assessed includes Pass, Fail and Not gradable. Gradable includes Pass and Fail. Both use the pinned test count in that published Result.
Out of 15 scenarios
The summary counts each of the 15 scenarios once. A scenario fails when any of its published test cases failed. “No published Result yet” means nothing is public for that scenario; it does not say whether Edcafe AI was tested on it.