Privacy
Last updated: 2026-10-09
What we store today
- Your account: your email address and a hashed password, managed by Supabase Auth.
- Your projects (name only).
- Details of your API keys: we store only a one-way hash of each key, so we cannot show it again, plus its name, a short display prefix of the key (the first 13 characters, which cannot be used to send events), its creation time and last-use time.
- The scores and event details you send to a project.
- When you sign in from the command line, the sign-in request: a one-time code (stored only as a hash), the device name and operating system the tool reports, and the time. It is kept for up to one day, then deleted.
- A security log of actions such as creating or revoking a key and deleting your account. The deletion entry is stored without your identity. Earlier log entries keep a random identifier that no longer links to you after you delete your account.
The local agent
The local agent (installed with pip) sends us only the scores it computes, with small metadata such as which signal, when, and which embedding model. Prompts are never read. The text of a response stays on your machine: it is held in memory long enough to compute a score and is never written to disk or sent to us.
The optional judge is off until you turn it on. When it is on, about one in 25 responses is redacted on your machine (emails, API keys, tokens and long numbers are masked, and the text is cut to 1500 characters) and sent for grading: to the model provider you chose, using your own key, or, if you use the free allowance, to our relay. The relay forwards it to a language-model provider (currently Groq), keeps nothing, and logs only a status code and a duration. We store a count of how many graded samples your account has used, never their content.
Connected models
If you connect a model from the website, you give us your own provider API key. It reaches our server over an encrypted connection, is checked once with your provider, and is then encrypted with a key that only our scheduled runner holds, so our website and database cannot read it afterwards. The runner opens it only while it runs your fixed questions, and it is never written to a log. We store the encrypted key, its last four characters, your schedule, your probe questions, and numbers derived from the answers (never the answers themselves). You can delete all of it at any time on the connect page; deleting your project or account removes it too. Every question is a call on your provider account and is billed by your provider.
How long we will keep scores
Raw scores will be kept for 30 days, then rolled up into hourly summaries.
The live demo
The shared demo sends the question you type through the demo server to an LLM provider. Do not type anything sensitive into it.
Bot check
The sign-in and sign-up forms may use Cloudflare Turnstile, a bot check. It processes limited browser data under Cloudflare's terms.
Cookies and analytics
The site sets only cookies needed for signing in (the session and a short-lived sign-in verifier). It sets no third-party analytics at launch.
Where
Supabase hosts the database and sign-in. Vercel hosts the website. Render hosts the demo API.
Deleting your data
Contact
Write to singh.rajvardhan.it@gmail.com. The same contact is in security.txt.