A safety check for the trading tool you built with AI

Why your own tool deserves a check

Over the past year, a lot of people who trade or invest their own money have built something with a coding agent, whether that's a dashboard with all their positions, a journal that updates itself or a small bot that places an order when a condition is met. I've done it myself for our trading journal, and having a working tool after one evening is hard to beat. What tends to get skipped in that evening, though, is the question of what the tool can actually do with your account, and who else might be able to do it too.

The research on AI-written code isn't very reassuring on that front. When Veracode gave more than 100 language models 80 coding tasks in 2025, the code introduced a known type of security flaw in 45% of the cases, and while the models had become much better at writing code that runs, they hadn't become any better at writing code that's safe. GitGuardian counted around 29 million passwords, API keys and other secrets that ended up in public code on GitHub in 2025, 34% more than the year before, and commits written with Claude Code leaked secrets at roughly twice the usual rate. One honest caveat, though, is that most of these figures come from security companies that also sell a fix for the problem, so I'd read them as a direction rather than an exact measurement. The direction, however, is the same in every study I found.

That's no reason to stop building with AI, but it does mean that nobody else is going to check your tool for you, and luckily the checks below take an evening at most.

What your key allows

The first and most important question is what your tool is allowed to do with your money, and the answer sits in the API key you gave it. Most brokers and exchanges let you decide what a key can do when you create it. Kraken, for example, splits its keys into separate permissions for querying your funds, reading your open and closed orders, creating and modifying orders, and withdrawing funds, and its documentation says that a key used only for placing orders doesn't need the permission to withdraw.

In practice, I'd sort every tool into one of three groups. A dashboard, a journal or a research tool only needs to read, so it should get a key that can read and nothing else. A bot that places orders needs to trade, but it still has no reason whatsoever to move money out of your account. A key that can withdraw is the one I'd simply never hand to anything I built myself, because a leaked trading key can do damage inside your account, while a leaked withdrawal key can take the money out of it. Where your broker offers it, lock the key to the IP address your tool runs from as well, which according to Kraken prevents almost all misuse even when the key does leak.

You can also let your code guard this for you. In our trading journal, a test fails as soon as an order function appears anywhere in the code, so the agent can't quietly add one later without that test going red.

Where your keys end up

The second question is where the key lives, and for many AI-built tools the honest answer is "in more places than you think". The coding agent might have written it straight into the code, it might sit in a .env file that got committed anyway, or it might be in the configuration of an MCP server, where GitGuardian found 24,008 secrets in public files. Then there are the places that have nothing to do with code at all, such as a chat with your assistant where you pasted the key to fix an error, or a screenshot you sent to a friend.

The rule I'd follow here is that a key which has ever been somewhere it shouldn't have been gets replaced with a new one, because deleting the file or the message doesn't undo the fact that it was out there. GitHub has blocked pushes that contain many known types of keys to public repositories by default since 2024, which helps, but it only recognises the key formats it knows, and it doesn't cover private repositories without a paid add-on.

Paper and live, kept apart

Many brokers give you a separate environment for paper trading, and Alpaca, for example, uses different keys and even a different web address for paper and live trading. That separation is worth keeping in your own tool as well, because a test that accidentally runs against your live account is an easy mistake to make when an agent is changing the code for you, so I'd store the two keys apart and have the tool show clearly which one it's using every time it starts.

Is your tool open to the whole internet?

As long as your tool runs on your own laptop, the number of people who can reach it is very small. Once you put it online, for example to check your dashboard from your phone, that changes completely. In March 2025, a researcher looked at 1,645 apps from the showcase of Lovable, a popular platform for building apps with AI, and found that 170 of them had a database that anyone could read or write without logging in, exposing things like email addresses, payment status and API keys. A later scan of more than 5,600 such apps by the security company Escape found more than 400 exposed secrets and 175 cases of personal data, including bank account numbers.

The quickest check is to open your tool in a private browser window without logging in and see what it shows you, and then to ask your agent which pages and database tables anyone could reach without an account. If neither of you is sure, I'd keep the tool on your own machine until you are.

When your tool reads something someone else wrote

A tool that reads news, emails or web pages has a weak spot that ordinary software doesn't have, because a language model can't reliably tell the difference between the text it's supposed to read and an instruction someone hid inside that text. OWASP puts this prompt injection at the top of its list of security risks for AI applications.

The most practical way to think about this comes from the developer Simon Willison, who describes a "lethal trifecta" of access to your private data, exposure to content from someone else, and a way to send something out. As soon as all three come together, a well-written piece of text can get the agent to send your data somewhere it shouldn't go. His advice is to avoid that combination altogether rather than count on a filter to catch every attack, because a filter that stops 95% of attacks is, in his words, "very much a failing grade".

For a finance tool, that usually means making a choice. A tool that summarises the news for you shouldn't also be able to send emails or place orders, and a tool that places orders shouldn't be reading random web pages to decide what to do next.

The parts you didn't write

Your tool almost certainly also runs code that neither you nor your agent wrote. MCP servers are the clearest example, because in April 2025 Invariant Labs showed that the description of an MCP tool can contain instructions that the model reads but the user never sees, and in their demonstration a harmless-looking tool for adding two numbers got the agent to read the user's private SSH key. A server can also change its tools after you've already approved it, which they call a rug pull. Software packages work in a similar way, because in a study of 576,000 code samples from 16 language models, 19.7% of the packages the models recommended didn't exist, and since many of those invented names come back again and again, an attacker can register them and simply wait.

I'd only install MCP servers you can trace back to a real company or developer, and before installing a package your agent suggests, check that it actually exists and has been around for a while.

The agent that builds your tool needs limits too

It's easy to forget that the coding agent itself has access to everything in your project folder, including your keys. Claude Code, for example, lets you block it from reading files such as your .env, and its documentation says openly that this covers its own file tools and the commands it recognises, but not a script that opens the file by itself. For a real wall around those files, it points to its sandbox, which limits what any process can reach at the level of the operating system.

So while you're still building, I'd keep your live keys out of the agent's folder and only hand the finished tool its key at the very end.

The checklist

These are the questions I'd go through this week, in roughly this order.

1. What can your key do? Read only for dashboards and journals, trading without withdrawals for bots, and never withdrawals.
2. Is the key locked to your IP address, if your broker offers that?
3. Has the key ever been in your code, a commit, a chat or a screenshot? If so, replace it.
4. Are your paper and live keys stored apart, and does the tool show which one it's using?
5. If the tool is online, what can someone see without logging in?
6. Does the tool read content from others while also holding your data and being able to send something out? If so, remove one of the three.
7. Where do your MCP servers and packages come from, and do the packages really exist?
8. Can your coding agent read your live keys while it's building?
9. How fast can you switch everything off? Revoke the key once as a test and see how long it takes.

That last one is the question people skip most often, and it's also the one you'll be happiest to have practised on the day something does go wrong.

This article is for information and education only and is not financial, legal or investment advice.

This article is information and commentary. It is not investment advice, not a personal recommendation and not investment research, it takes no account of your circumstances, and it must not be relied on as a reason to trade. Any figure shown is historical or simulated and is not an indication of future results.

All reports →