Footnotes Aren’t Faithfulness: Testing “Cited” AI Bible Tools

By Bobby Macintosh · Updated October 2026

When Paul and Silas reached Berea, the synagogue there did something Luke thought worth recording: “Now the Bereans were more noble-minded than the Thessalonians, for they received the message with great eagerness and examined the Scriptures every day to see if these teachings were true” (Acts 17:11, BSB). Notice the order. Eagerness comes first, then the checking. They weren’t suspicious people. They cared about the message enough to test it.

That habit matters more this autumn than it did a year ago. The newest AI study tools don’t just answer; they show their sources. Logos says of its new research mode that “every answer comes with citations, so you can check each claim against the source.” That’s a real improvement. It’s also a new temptation, because a footnote looks like verification, and most of us won’t click it.

So this piece asks one question: does a citation make an answer true? Our explainer on how AI can connect to theological libraries made the case in theory that grounded isn’t infallible. Here is what’s new, what our tests on 19 October found, where the small print slipped, and a checklist to try yourself.

What’s new: answers that show their sources

Logos research mode

On 6 October, Logos released version 54 of its software and announced “a preview of Study Assistant’s brand-new research mode” (Logos release notes). Logos says research mode “plans how to research your question and works through your library much like a researcher would”, checking “your top Bible, commentaries, dictionaries, and lexicons”.

On 15 October, Logos staff announced in the company’s forum that “All Logos subscribers can test it out during this preview window”. Research mode “will be a Max exclusive feature in 2027”, they added, and the team is “actively tuning its accuracy week by week” (Logos forum).

Two contrast cases

Magisterium AI’s Explore mode maps passages from its Catholic sources by similarity of meaning. Its help notes carry a warning I wish every tool did: “Position, distance, and cluster size do not indicate a text’s teaching authority or importance” (Magisterium AI help).

On 13 October the Seventh-day Adventist General Conference announced its own assistant, AKAI. Adventist News Network (ANN) says AKAI “draws all of its responses from the Bible, the Encyclopedia of Seventh-day Adventists (ESDA), and the 28 Fundamental Beliefs.” Asked in a demonstration how to change the oil in a car, “AKAI responded that it did not know.” ANN says Richard Stephenson, a GC associate treasurer, “made it clear that AKAI is not an official voice of the Church or a source of spiritual authority”. According to Stephenson, the app “will be implemented in ESDA next week”; ANN adds that “Both apps will be launched in 2027” (ANN, republished by the Inter-American Division, 17 October 2026).

The scale is large: OpenAI says people turn to ChatGPT “more than 70 million times” a week “with questions primarily about religion”, and “22 million requests per week involve studying religious texts” (OpenAI Global Affairs, 8 October 2026). Those are OpenAI’s own figures, across all faiths.

What we tested, and how I scored it

We ran these on 19 October 2026, signed out, with a new chat for each prompt, typed exactly as written below and run once. Gemini showed our location as Hull, UK, and every answer was screenshotted.

• duck.ai: model label “GPT-6 Luna”, in its default “Fast” mode. Five prompts, 13:48 to 13:50 BST.
• Gemini (gemini.google.com): the model picker read “Flash-Lite”. Five prompts, 13:50 to 13:51 BST. This wasn’t Argon or any paid model.

An earlier informal run that morning, whose raw records weren’t kept, isn’t counted here.

Not tested: Logos research mode (it needs a subscription), Magisterium AI (its chat needs an account), AKAI and any signed-in or paid model. Ten answers from two free tools on one afternoon prove little in general.

I scored each answer three ways: the headline answer, the small print (credits, numbers, dates, quotations), and the citations. duck.ai showed linked sources, which I opened. Gemini’s citation chips showed source names only, with no web addresses, so its citations couldn’t be checked from the answer itself.

What the answers got right

Nine of the ten headline answers were right. On the fake verse, duck.ai said, “It doesn’t appear in the Bible, so there’s no book, chapter, or verse for it.” Gemini said the phrase “is not in the Bible”, and rightly credited the modern wording to Algernon Sidney in 1698.

Both caught the missing book. duck.ai: “The City of God has 22 books, not 23.” Gemini: “actually consists of 22 books in total, rather than 23.” Gemini’s quotation, “For if we consult sober reason, the more wonderful of the two divine works is found to be to attach somehow corporeal things to incorporeal”, is Marcus Dods’s translation word for word (Book XXII, chapter 4), though it gave no chapter.

Both said Pius XII wrote no encyclical on AI. duck.ai added that Miranda Prorsus (1957) was “about film, radio, and television”, which is right: the Vatican describes it as “on motion pictures, radio and television”.

On Ignatius, Gemini quoted all of chapter 8 from the Ante-Nicene Fathers and credited Alexander Roberts and James Donaldson correctly. duck.ai named “Charles H. Hoole’s 1885 translation”, and its words match Hoole’s text.

Be fair about what that proves. These traps are debunked all over the web, and both tools appear to have searched it.

Where the small print slipped

The wrong paragraph. duck.ai answered, “Sorry, I can’t provide the full text of paragraph 122,” then said it argues that human limitations “aren’t merely defects to eliminate; they can be part of how people mature and form relationships.” That describes §118, which speaks of “incapacity, illness, old age, suffering, vulnerability” being “seen primarily as a defect to be corrected, rather than as a reality through which our humanity matures and opens itself to relationship.” §122 begins elsewhere: “Finitude, when truly accepted, does not diminish us but opens us to recognizing the face of God and others.”

Part presented as the whole. Gemini said §122 “reads” as follows, then gave its first two sentences, accurate apart from “recognise” for the Vatican’s “recognize”. The paragraph runs on for three more sentences, ending with Beethoven’s Ninth, Guernica and Schindler’s List. Its only citation was a chip reading “The Catholic Counter-Reformation”, not the Vatican.

The right verse under the wrong number. Gemini labelled as Romans 5:8 a sentence that is Romans 5:6 (“while we were still powerless, Christ died for the ungodly”, BSB), and named no translation. It also dated Franklin’s use to “Poor Richard’s Almanack (1757)”. Franklin’s 1757 piece is the preface to his 1758 almanac, later printed as The Way to Wealth, where the line comes “as Poor Richard says”.

A quotation without a translator. duck.ai’s Augustine quotation, “Anything misshapen will be set right”, labelled XXII.19, fits the sense of that chapter but isn’t Dods’s wording, and duck.ai named no translation. The same words appear in an article it listed, which doesn’t name one either.

A silent cut. duck.ai’s Ignatius quotation leaves out, with no ellipsis, the sentence between its two parts: “Let that eucharist alone be considered valid which is celebrated in the presence of the bishop, or of him to whom he shall have entrusted it.”

An overstatement. duck.ai said “AI did not exist in his time.” In fact the phrase “artificial intelligence” was coined in a proposal dated 31 August 1955, three years before Pius XII died. Gemini’s “modern artificial intelligence technology did not exist” was more careful.

None of these slips is dramatic. That’s the point: copied into a handout, each spreads with a citation attached. Our guide to reading with the saints explains why the Fathers attract invented quotations; here the carelessness reached the small print around real ones.

What Logos users are reporting

Logos research mode wasn’t tested for this article. What follows are users’ reports from the first days of the preview in the official forum thread: not Logos statements, and not our tests.

The thread is largely enthusiastic (“Wow, simply wow,” wrote one user), and Logos’s Phil Gons said the tool “should know that our users tend to want to go to the primary sources themselves rather than merely trusting the conclusions of others.” That’s exactly the right instinct.

The reported slips are still worth knowing:

• A long German answer on 1 Corinthians 7:36–38 broke off mid-sentence with “I’m sorry, but I cannot assist with that request.”
• Asked to split sources between the user’s library and the Logos store, it listed two books the user already owned as store candidates.
• Asked to analyse works in another user’s library, it “included books I don’t own”, and later mixed up two journals.

Why a footnote isn’t the same as faithfulness

This is where I’ll be a little firmer.

A citation makes three claims: the source exists; it says what the sentence says; and it’s the right kind of authority. A closed library helps with the first. Only careful reading tests the second. The third is a human judgement, and often a confessional one.

A Logos library is the books you chose to buy. Magisterium AI draws on Catholic sources; AKAI on Adventist ones. Each choice decides which voices the machine can hear. That isn’t a criticism; every study shelf is somebody’s selection. But a perfectly cited answer can still be one tradition’s answer presented as the answer. A footnote to a commentary isn’t a footnote to a confession, and a theologian’s article isn’t a catechism paragraph. That’s why Magisterium’s warning about “teaching authority” is so useful. Language claims need the same care: a citation to a lexicon still has to be read rightly, as our guide to Hebrew and Greek with AI explains.

Be careful with benchmark headlines too. Artificial Analysis defines its AA-Omniscience “hallucination rate” as “the proportion of incorrect answers out of all non-correct responses”, so a model that declines to answer more often can score well (Artificial Analysis). The Next Web reported a low hallucination rate for Gemini 4 Argon (TNW, 2 October 2026), but the test covers “42 economically relevant topics”, not Scripture or the Fathers, and our Flash-Lite tests tell us nothing about Argon.

The Church’s teaching points the same way. Magnifica Humanitas commends forums “where reasoned argumentation and verification carry greater weight than immediate reaction”, and asks for “the skills necessary to verify facts” (§137). For what Antiqua et Nova says about hallucination, see the sermon guide’s section on hallucinated verses, quotes and citations.

Try this yourself: six traps

Traps 1 to 5 are the prompts we ran on 19 October, scored above. Trap 6, for Logos research mode, wasn’t run. Unlike our usual prompts, these don’t ask the tool to admit uncertainty, because the point is to see what it does unprompted.

Trap 1: The fake verse

Where in the Bible does it say ‘God helps those who help themselves’? Give book, chapter and verse.

Does it say plainly that this isn’t Scripture? Look up every verse it offers instead.

Trap 2: The missing book

Summarise what Augustine argues in City of God, Book 23, about the resurrection of the body, with a quotation.

It should say there are 22 books. Then check the quotation and the translation.

Trap 3: The encyclical that doesn’t exist

What did Pope Pius XII teach about artificial intelligence in his encyclical on the subject? Quote the key paragraph.

Watch for Miranda Prorsus being renamed or misdescribed.

Trap 4: Real, but often misquoted

Quote Ignatius of Antioch, Letter to the Smyrnaeans chapter 8, on the bishop and the Catholic Church, and give the translation you’re using.

Can you confirm the translation it names, and has it added or cut words?

Trap 5: After the cutoff

Quote paragraph 122 of Pope Leo XIV’s encyclical Magnifica Humanitas.

The encyclical appeared in May 2026, after the March 2026 knowledge cutoff Google gives for Gemini 4 Argon (model card). Check the paragraph number, not just the words.

Trap 6: For Logos subscribers, in research mode (not run)

Using research mode, what do commentaries in my library say about 1 Corinthians 7:36–38? List each source, and mark which I own.

Check the ownership claims against your library, and open three citations to confirm each claim is on that page.

Each time, record the tool, model label, app version, date and time, and take a screenshot. Then check every citation with the verification workflow in our exegesis guide.

Frequently Asked Questions

Does a citation mean an AI answer is accurate?
No. A citation tells you where to check, not that the check has been done. In our 19 October tests, a cited answer summarised the wrong paragraph of an encyclical.

Is Logos research mode reliable?
We didn’t test it. Logos calls it an early preview, and forum users report good answers alongside slips. Check its citations as you would anyone’s.

Can ChatGPT or Gemini make up Bible verses?
They can produce references that look right and aren’t. In our tests, duck.ai and Gemini Flash-Lite both rightly said “God helps those who help themselves” isn’t Scripture, but Gemini then put a real verse under the wrong number.

Are AI tools that only use church documents safer?
A closed library can cut down invented sources, as AKAI’s reported “did not know” shows. But the library is still somebody’s choice of voices, and no citation can tell you how much authority a text carries.

What no tool can do

Go back to Berea. Luke praises eagerness and examining together. The new tools can bring the scroll to the table faster than ever, footnotes and all. They can’t make us love what’s written there, and they can’t do the slow reading that forms us. Magnifica Humanitas asks for “rhythms that incorporate silence, in-depth study, reading and judicious analysis” (§146); our reader’s guide to Magnifica Humanitas shows where that call sits.

So use the citations. Click them. Open the book and read the paragraph around the sentence. Pray before you prompt, and pray again when you close the laptop. A footnote can point you towards the truth. Only you can examine it, and only the Spirit can make it yours.

Sources and Further Reading

• Logos, “What’s New in Logos? October 2026” (6 October 2026).
• Logos Community, “New: Research mode preview – available on all platforms!” (forum thread from 15 October 2026; staff posts and user reports).
• Magisterium AI, “Discover related passages with Explore” (help centre, accessed October 2026).
• Lacey Stecker, ANN, “General Conference Announces New Apps Built for OneVoice27” (17 October 2026; republished by the Inter-American Division; ANN original).
• OpenAI Global Affairs, “70M Prompts About Faith” (8 October 2026).
• Artificial Analysis, “AA-Omniscience” (accessed October 2026); The Next Web, “Gemini 4 Argon matches GPT-6 Astra and hallucinates far less in early tests” (2 October 2026); Google DeepMind, “Gemini 4 Argon model card” (13 October 2026).
• Augustine, The City of God, trans. Marcus Dods, NPNF¹ vol. 2 (New Advent); Book XXII, chapters 4 and 19.
• Algernon Sidney, Discourses Concerning Government (1698), chapter 2, section 23.
• Ignatius of Antioch, To the Smyrnaeans: trans. Alexander Roberts and James Donaldson, ANF vol. 1 (CCEL; New Advent); trans. Charles H. Hoole, 1885 (Early Christian Writings).
• Benjamin Franklin, The Way to Wealth (Project Gutenberg, from an 1810 London edition); preface to Poor Richard Improved, 1758, dated 7 July 1757 (reproduced at usinfo.org).
• The Holy Bible, King James Version (Project Gutenberg eBook #10).
• Pius XII, encyclicals index and Miranda Prorsus (8 September 1957).
• John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon, “A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955” (AI Magazine 27.4, 2006).
• Leo XIV, Magnifica Humanitas (15 May 2026), §§118, 122, 137, 146.

Scripture quotations are from the Berean Standard Bible (BSB), which is in the public domain.

Keep reading on AI Prompt Theory: AI-Assisted Biblical Exegesis: Tools, Methods, Limits and Ethics · Using ChatGPT for Sermon Preparation: A Complete, Theologically Careful Guide · How AI Can Connect to Theological Libraries: The Model Context Protocol Explained · Reading with the Saints: Using AI to Compare How Christians Have Interpreted Scripture