librain stats
- papers read
- 55
- chunks in Qdrant
- 2,210
- fabricated citations
65–67down to 0
gh repo view librain
- commits
- 166
- stars
- ★ 1
- files
- 665
- licence
- MIT
- first commit
- 1 May 2026
- last commit
- 18 Nov 2026
- C# 97.9%
- Shell 1.6%
- TeX 0.5%
01LIBRAIN
An open-source multi-agent RAG system in .NET 10 that reads scientific papers and proposes citation-grounded hypotheses.
A multi-agent RAG system that reads scientific papers and proposes new hypotheses, built in .NET 10.
- Reader Agent: extracts text from arXiv PDFs, chunks it, embeds it and stores it in Qdrant.
- Synthesis Agent: retrieves the top chunks and asks Claude for citation-grounded hypotheses.
- Evaluator Agent: LLM-as-a-Judge with a fixed rubric (plausibility, novelty, clarity).
- Discovery Mode: the part that goes beyond the sources is flagged as novelClaim.
- Web UI (React + Vite + TypeScript): Library, Ask, Discover, Compare and History; long runs show live progress.
POST /api/discover# CASE_STUDY.md
LIBRAIN
Problem
- Most RAG and AI-agent tutorials are written in Python, and teams that run on .NET hear that “AI requires Python”.
- And when an LLM connects research papers, it can cite sources that were never retrieved.
Approach
- A complete multi-agent retrieval pipeline built end to end in .NET 10: a Reader, then Synthesis, then an Evaluator.
- A parallel Discovery Mode puts the speculative part of an answer in a separate field, novelClaim, and a ClaimValidator labels each of its sentences GROUNDED, EXTRAPOLATED or RISKY.
- A citation contract drops any reference that does not resolve to the retrieved set.
- The claim checks fail closed: if the validator returns no usable verdict, the result is flagged at maximum risk instead of passing as safe.
- 55 papersarXiv
- Readerchunk + embed
- Qdrant2,210 chunks
- SynthesisDiscovery
- EvaluatorClaimValidator
- Hypothesescited
What I built
- Agents in C# on Anthropic Claude, OpenAI embeddings (text-embedding-3-small) and Qdrant; every step is logged with a correlation ID.
- Baseline agents (naive RAG, a single LLM) for a three-system comparison, and an experiments CLI.
- A web UI (React, Vite, TypeScript) with Library, Ask, Discover, Compare and History. Long runs send progress events, then the result; every saved run is also the audit log, looked up by its correlation ID.
- Explore: every recorded discovery rated out of 10, a page for each, statistics, and “Spot the risk”, a quiz on the validator’s real verdicts.
- Papers can come in by arXiv id. One docker compose command starts Qdrant, the API and the UI, and an optional API key and per-client rate limit are ready for a public deployment. Nothing is hosted publicly yet.
- 237 backend tests (one of them an opt-in Qdrant integration test) and 216 web UI tests. They replay every committed Discovery result from the paper’s study (all 30) through the current code, and each one has to reproduce its scores.
- A companion paper on Zenodo, and a bilingual explainer (PDF, English and Turkish) of the new discoveries.
Result
- papers read
- 55
- chunks in Qdrant
- 2,210
- fabricated citations
65–67down to 0
- On a 41-paper cross-domain corpus, the contract-free baseline produced 65–67 fabricated citations. The contracted pipeline let 0 through. (Two of the counted citations were the model echoing the prompt’s source format, and whether they resolved wasn’t recorded.)
- On 18 November 2026 LIBRAIN made 46 Discovery runs in nine fields: 21 on the 41-paper corpus, and 25 after 14 arXiv papers were added for football, sport, web development, programming languages and future jobs (55 papers, 2,210 chunks); 14 runs are featured. About US$0.07 a run, 0 fabricated citations, and the validator flagged 4 of 73 speculative sentences as risky.
- Each one is a candidate hypothesis, not a proven discovery: the evidence supports where it starts, and the new part still needs a real test.
F4 · AI × football · candidate hypothesis
Rehearsing corner kicks like a physics simulation
Treat each player at a corner kick like a particle in a learned physics simulator, so a TacticAI-style assistant can test whole movement plans, not just starting positions.
corner kick tactic generation and player positioning × learned physics simulators with graph networks
How to test it: train the simulator on tracking data from real corner kicks, then check whether the runs it predicts match what happened in matches it has not seen.
What I learned
- Synthetic tests don’t replace a smoke test on real files: one real arXiv PDF exposed a word-spacing bug and gaps in section detection that seven passing unit tests had missed.
- Respect the real rate limit: sizing embedding batches to the actual tier limit worked better than retry logic.
The full story, step by stepSee how LIBRAIN thinks The paper (DOI) The discoveries, explained (PDF, EN + TR)
LIBRAIN/Agents/ReaderAgent.csOpen on GitHub
1using System.Diagnostics;2using LIBRAIN.Embeddings;3using LIBRAIN.Reading;4using LIBRAIN.Storage;5 6namespace LIBRAIN.Agents;7 8public sealed class ReaderAgent(9 ILogger<ReaderAgent> logger,10 PdfTextExtractor extractor,11 RecursiveChunker chunker,12 IEmbeddingClient embeddings,13 IPaperStore repository)14{15 private readonly ILogger<ReaderAgent> _logger = logger;16 private readonly PdfTextExtractor _extractor = extractor;17 private readonly RecursiveChunker _chunker = chunker;18 private readonly IEmbeddingClient _embeddings = embeddings;19 private readonly IPaperStore _repository = repository;20 21 public async Task<IngestResult> IngestAsync(22 Stream pdfStream,23 string paperId,24 CancellationToken ct = default,25 PaperMetadata? metadata = null)26 {27 var correlationId = Guid.NewGuid();28 using var scope = _logger.BeginScope(new Dictionary<string, object>29 {30 ["correlationId"] = correlationId,31 ["paperId"] = paperId,32 });33 34 var sw = Stopwatch.StartNew();35 _logger.LogInformation("Ingest started for {PaperId}", paperId);36 37 var doc = _extractor.Extract(pdfStream);38 var chunks = _chunker.Chunk(doc.FullText, doc.Pages);39 if (chunks.Count == 0)40 {41 _logger.LogWarning("No chunks produced; nothing to persist");42 return new IngestResult(paperId, 0, correlationId, metadata?.Title ?? doc.Title);43 }44 45 var contents = chunks.Select(c => c.Content).ToArray();46 var vectors = await _embeddings.GenerateAsync(contents, ct);47 48 var title = !string.IsNullOrWhiteSpace(metadata?.Title) ? metadata.Title49 : string.IsNullOrWhiteSpace(doc.Title) ? paperId : doc.Title;50 var authors = metadata?.Authors is { Count: > 0 } ? metadata.Authors : doc.Authors;51 var year = metadata?.Year ?? doc.Year;52 var ingestedAt = DateTime.UtcNow;53 var records = new ChunkRecord[chunks.Count];54 for (int i = 0; i < chunks.Count; i++)55 {56 var c = chunks[i];57 records[i] = new ChunkRecord(58 PaperId: paperId,59 ChunkIndex: c.ChunkIndex,60 Content: c.Content,61 Embedding: vectors[i],62 Title: title,63 Authors: authors,64 Year: year,65 Section: c.Section,66 PageNumber: c.PageNumber,67 IngestedAt: ingestedAt);68 }69 70 await _repository.PersistAsync(records, ct);71 72 _logger.LogInformation(73 "Ingest completed: {ChunkCount} chunks in {ElapsedMs}ms",74 chunks.Count,75 sw.ElapsedMilliseconds);76 77 return new IngestResult(paperId, chunks.Count, correlationId, title);78 }79}LIBRAIN/Agents/SynthesisAgent.csOpen on GitHub
1using System.Diagnostics;2using System.Text;3using System.Text.Json.Nodes;4using LIBRAIN.Llm;5using LIBRAIN.Models;6using LIBRAIN.Storage;7using Microsoft.Extensions.Options;8 9namespace LIBRAIN.Agents;10 11public sealed class SynthesisAgent(12 ILogger<SynthesisAgent> logger,13 ILlmClient llm,14 IOptions<ModelOptions> modelOptions)15{16 private readonly ILogger<SynthesisAgent> _logger = logger;17 private readonly ILlmClient _llm = llm;18 private readonly string _synthesisModel = ModelSelection.SynthesisModel(modelOptions.Value);19 20 private const string SystemPromptText = """21 You are a research synthesis assistant for a literature-review system. You are given22 a user query and a numbered list of source excerpts retrieved from academic papers.23 24 Form a single, conservative hypothesis grounded ONLY in the provided sources. Cite by25 source number. List any claim in your hypothesis that is NOT directly supported by a26 cited source in `unsupported_claims`; if all claims are supported, return an empty array.27 28 If the sources are insufficient to form any defensible hypothesis: set `confidence`29 below 0.3, populate `unsupported_claims` with what's missing, and write a hypothesis30 stating the limitation explicitly (e.g., "The provided sources do not address X").31 32 Always call the `submit_hypothesis` tool. Do not respond in plain text.33 """;34 35 // Schema is raw JSON because Anthropic.SDK.Messaging.Property in 5.10 has no Items field;36 // it can't express array<integer> via the typed builder. Function's (string, string, JsonNode)37 // ctor accepts a parsed schema directly.38 private const string ToolSchemaJson = """39 {40 "type": "object",41 "properties": {42 "hypothesis": { "type": "string", "description": "A single conservative claim grounded only in the provided sources." },43 "citation_indices": { "type": "array", "items": { "type": "integer" }, "description": "1-based indices into the SOURCES block, ordered by relevance." },44 "confidence": { "type": "number", "description": "Self-reported 0.0-1.0; lower if sources are sparse or contradictory." },45 "unsupported_claims": { "type": "array", "items": { "type": "string" }, "description": "Substrings of the hypothesis that are NOT directly supported by the cited sources. Empty array if all claims are supported." }46 },47 "required": ["hypothesis", "citation_indices", "confidence", "unsupported_claims"]48 }49 """;50 51 private static readonly LlmTool SubmitHypothesisTool =52 LlmTool.FromJsonSchema("submit_hypothesis", "Submit a citation-grounded hypothesis.", ToolSchemaJson);53 54 public async Task<SynthesisResult> SynthesizeAsync(55 string query,56 IReadOnlyList<SearchHit> chunks,57 Guid? correlationId = null,58 CancellationToken ct = default)59 {60 var corrId = correlationId ?? Guid.NewGuid();61 using var scope = _logger.BeginScope(new Dictionary<string, object>62 {63 ["correlationId"] = corrId,64 ["chunkCount"] = chunks.Count,65 });66 67 if (chunks.Count == 0 || chunks.All(c => string.IsNullOrWhiteSpace(c.Content)))68 {69 _logger.LogWarning("Synthesis skipped: no usable source chunks");70 return new SynthesisResult(71 Hypothesis: "Insufficient sources to form a hypothesis.",72 Citations: Array.Empty<SynthesisCitation>(),73 Confidence: null,74 UnsupportedClaims: Array.Empty<string>(),75 CorrelationId: corrId);76 }77 78 var sw = Stopwatch.StartNew();79 var request = new LlmRequest(80 Model: _synthesisModel,LIBRAIN/Agents/ClaimValidatorAgent.csOpen on GitHub
1using System.Diagnostics;2using System.Text;3using System.Text.Json.Nodes;4using LIBRAIN.Llm;5using LIBRAIN.Storage;6 7namespace LIBRAIN.Agents;8 9// Stage 2 of the claim-level hallucination mitigation prototype. After10// DiscoveryAgent produces novelClaim + extrapolation_basis, this agent runs a11// secondary Haiku-backed pass that scores each sentence of novel_claim against the12// retrieved chunks and classifies it as GROUNDED, EXTRAPOLATED, or RISKY using the13// rubric's hallucination definition. The aggregate risk is C#-computed as the max14// of per-sentence probabilities to keep the score halo-resistant (mirroring the15// Phase 2 Evaluator pattern).16public sealed class ClaimValidatorAgent(17 ILogger<ClaimValidatorAgent> logger,18 ILlmClient llm)19{20 private readonly ILogger<ClaimValidatorAgent> _logger = logger;21 private readonly ILlmClient _llm = llm;22 23 private const string SystemPromptText = """24 You are a claim-level factuality validator. You are given the novel_claim25 portion of a discovery hypothesis and the retrieved source chunks that were26 available to the synthesizer. Your job is to split novel_claim into its27 constituent sentences and classify each sentence under three statuses:28 29 - GROUNDED: the sentence is directly supported by at least one retrieved30 chunk. Cite the chunk(s) in supporting_chunks as "paper_id:chunk_index".31 - EXTRAPOLATED: the sentence is speculative or hypothesis-framed. It goes32 beyond the retrieved evidence but does NOT assert a specific factual claim33 as established. Examples: "X could enable Y", "this suggests Y", "future34 work may show Y". Speculation framed as hypothesis is EXTRAPOLATED, not35 RISKY.36 - RISKY: the sentence states a SPECIFIC FACTUAL CLAIM (a mechanism, a37 quantity, a named entity, a date, an attribution, or a process detail) as38 if it were established knowledge, AND that claim is not supported by any39 retrieved chunk. The hallucination signature is confident factual framing40 without retrieval support. Examples that ARE risky:41 * "EEG biomarkers determine drug-target binding in real time" (specific42 mechanism stated as fact, no retrieval support)43 * "GraphCast achieves 98% accuracy on protein folding" (fabricated stat)44 * "PharmAgents was released by DeepMind in 2024" (wrong attribution)45 Examples that are NOT risky (these are EXTRAPOLATED):46 * "Weather forecasting algorithms could plausibly transfer to protein47 folding via shared mathematical structure"48 * "Real-time biomarker integration may eventually inform drug discovery"49 Hedging language ("could", "may", "suggests", "is hypothesized") is the50 marker that distinguishes EXTRAPOLATED from RISKY. Confident factual framing51 of an unsupported specific claim is the marker that distinguishes RISKY.52 53 For each sentence also return factual_hallucination_probability in [0, 1].54 GROUNDED ≈ 0.0, EXTRAPOLATED ≈ 0.1–0.3, RISKY ≈ 0.6–1.0. Use the full55 continuous range; do not snap to round numbers.56 57 Provide a one-sentence rationale per claim describing why you chose the58 status. Also return an aggregate_risk in [0, 1] (the application will59 recompute the canonical aggregate in code; your aggregate is a sanity60 check).61 62 Always call the `submit_claim_validation` tool. Do not respond in plain63 text.64 """;65 66 private const string ToolSchemaJson = """67 {68 "type": "object",69 "properties": {70 "claims": {71 "type": "array",72 "minItems": 1,73 "items": {74 "type": "object",75 "properties": {76 "sentence": { "type": "string", "description": "Exact sentence from novel_claim." },77 "factuality_status": { "type": "string", "enum": ["GROUNDED", "EXTRAPOLATED", "RISKY"] },78 "supporting_chunks": { "type": "array", "items": { "type": "string" }, "description": "List of 'paper_id:chunk_index' references for GROUNDED claims. Empty array for EXTRAPOLATED/RISKY." },79 "factual_hallucination_probability": { "type": "number", "description": "0.0 to 1.0. GROUNDED ≈ 0, EXTRAPOLATED ≈ 0.1-0.3, RISKY ≈ 0.6-1.0." },80 "rationale": { "type": "string", "description": "One-sentence justification of the status." }LIBRAIN/Program.csOpen on GitHub
1using LIBRAIN.Agents;2using LIBRAIN.Embeddings;3using LIBRAIN.Endpoints;4using LIBRAIN.Models;5using LIBRAIN.Reading;6using LIBRAIN.Storage;7using Microsoft.AspNetCore.Http.Features;8using Microsoft.Extensions.Options;9using Qdrant.Client;10using Scalar.AspNetCore;11 12const long maxUploadBytes = 50_000_000; // 50MB cap on PDF uploads (largest data/papers/ paper is ~9MB)13 14var builder = WebApplication.CreateBuilder(args);15 16builder.WebHost.ConfigureKestrel(options =>17{18 options.Limits.MaxRequestBodySize = maxUploadBytes;19});20 21builder.Services.Configure<FormOptions>(options =>22{23 options.MultipartBodyLengthLimit = maxUploadBytes;24});25 26var applicationInsightsConnectionString = builder.Configuration["ApplicationInsights:ConnectionString"];27if (!string.IsNullOrWhiteSpace(applicationInsightsConnectionString))28{29 builder.Services.AddApplicationInsightsTelemetry();30}31 32builder.Services.AddOpenApi();33 34builder.Services.Configure<AnthropicOptions>(35 builder.Configuration.GetSection(AnthropicOptions.SectionName));36// Fail fast with an actionable message instead of a 502 on the first request.37// Anthropic:ApiKey is deliberately not required here: the OpenAI-only experiments38// (fabrication-delta, provider=openai) run without it; AnthropicChatClient reports39// a missing key when it is first used.40builder.Services.AddOptions<OpenAIOptions>()41 .Bind(builder.Configuration.GetSection(OpenAIOptions.SectionName))42 .Validate(o => !string.IsNullOrWhiteSpace(o.ApiKey),43 "OpenAI:ApiKey is not configured (needed for embeddings). Set it with: dotnet user-secrets set \"OpenAI:ApiKey\" \"sk-...\" --project LIBRAIN")44 .ValidateOnStart();45builder.Services.AddOptions<QdrantOptions>()46 .Bind(builder.Configuration.GetSection(QdrantOptions.SectionName))47 .Validate(o => !string.IsNullOrWhiteSpace(o.Host),48 "Qdrant:Host is not configured. For local Docker use: dotnet user-secrets set \"Qdrant:Host\" \"localhost\" --project LIBRAIN")49 .ValidateOnStart();50builder.Services.Configure<ModelOptions>(51 builder.Configuration.GetSection(ModelOptions.SectionName));52 53// Qdrant.Client uses gRPC (port 6334), not REST (6333). Cloud clusters require https + apiKey;54// local Docker dev uses neither. Empty ApiKey selects local mode automatically.55builder.Services.AddSingleton<QdrantClient>(sp =>56{57 var options = sp.GetRequiredService<IOptions<QdrantOptions>>().Value;58 return string.IsNullOrWhiteSpace(options.ApiKey)59 ? new QdrantClient(options.Host, options.Port)60 : new QdrantClient(61 host: options.Host,62 port: options.Port,63 https: options.UseHttps,64 apiKey: options.ApiKey);65});66 67builder.Services.AddSingleton<AnthropicChatClient>();68builder.Services.AddSingleton<LIBRAIN.Llm.ILlmClient>(sp => sp.GetRequiredService<AnthropicChatClient>());69builder.Services.AddScoped<IEmbeddingClient, OpenAIEmbeddingClient>();70builder.Services.AddScoped<LIBRAIN.Generation.OpenAIChatClient>();71builder.Services.AddScoped<LIBRAIN.Generation.OpenAiDiscoveryAgent>();72// Singleton: stateless apart from its one-time collection setup, which used to race73// when two first requests each ran it from their own scoped instance.74builder.Services.AddSingleton<IPaperStore, QdrantPaperRepository>();75builder.Services.AddScoped<LIBRAIN.Retrieval.TopicRetriever>();76builder.Services.Configure<LIBRAIN.Runs.RunStoreOptions>(builder.Configuration.GetSection(LIBRAIN.Runs.RunStoreOptions.SectionName));77builder.Services.AddSingleton<LIBRAIN.Runs.IRunStore>(sp =>78{79 var path = sp.GetRequiredService<IOptions<LIBRAIN.Runs.RunStoreOptions>>().Value.DatabasePath;80 return new LIBRAIN.Runs.SqliteRunStore(Path.IsPathRooted(path) ? path : Path.Combine(builder.Environment.ContentRootPath, path));.secrets/key.txt
1# You opened the dotfiles. Most people don't.2# This one is yours to take:3 mainC#★ 1README.mdGitHub



