What to ask a product partner: the artefact list

We all know what a selection process looks like: a deck, two reference calls, a scoring matrix. How do you learn from any of that how a team behaves in month four, when the date has moved and somebody has to decide what gets cut?

Start with where I’m standing. I run product at a firm that builds for banks, so I’m not the one running these selections — I’m the one answering them. That’s a particular vantage point. You see which questions do real work, and which ones every supplier has a prepared slide for.

This is the list I’d want to be asked. Before publishing it I checked that we could produce every item on it ourselves. Ask us.

Answers describe how a supplier would like to work. Artefacts show how they worked last time.

There’s a second reason, specific to financial services. Accountability doesn’t move with the work. The FCA puts it plainly: a firm remains responsible and accountable for the regulatory responsibilities attached to its outsourcing and third party arrangements, and none of that can be delegated away (FCA, Outsourcing and operational resilience). The same holds for arrangements that fall outside the outsourcing definition — those still sit within the rules on governance, risk management, systems and controls. A design and engineering partner may well not be material outsourcing. You will still need to evidence what you relied on.

One date to hold. On 18 March 2026 the FCA published new operational incident and third party reporting requirements, with twelve months to prepare before they take effect on 18 March 2027 (PS26/2). Arrangements you enter into now will be documented under that regime.

Everything below already exists at a competent supplier, because they produced it while doing the work. Nothing here asks anyone to write something new for your pitch.

The people in the room are frequently not the people on the project. Undoubtedly the pitch team is the strongest team a supplier has — that is why they are in the room. Ask about the other one.

• Named delivery team, with current allocation. Who is assigned, at what percentage, and what else they carry during your dates. Eight weeks out, a supplier should be able to answer this on the call. If they can’t, ask again a week later and see whether the names changed.
• What the pitch team does after signature. Sometimes the answer is “nothing”, and that is fine — presenting and delivering are different jobs. Ask whether they stay reachable.
• Escalation and decision rights. Who decides when your stakeholders are on leave. Get it written down (one line in the SoW is enough).
• Turnover on the last two engagements of similar size. How many people changed mid-project, and why. Some churn is normal. Suppliers who track turnover will give you the number on the call. The ones who don’t will give you an adjective.
• Subcontracting. Which parts sit with another firm or with contractors. Onward subcontracting is a supervisory concern in its own right under PRA SS2/21, so an undisclosed chain becomes your problem.

You are buying your team’s ability to carry the work forward once the supplier is gone. That is hard to inspect in a pitch, so inspect the leftovers from somebody else’s project.

• A decision log. What was chosen, what was rejected, and why. If you ask for one thing on this list, ask for this one. Six months after the supplier has gone, this conversation happens somewhere in your team: “Why does the limit check run before the balance call?” — “I don’t know, it was like that when I joined.” — “Who decided it?” — “Someone at the agency, I think.” Either the reasoning was written down while it was fresh, or it gets reconstructed by people who weren’t there.
• A handover pack from a completed project. The actual material a previous client received on the final day. Redacted is fine.
• A design system as it is maintained. Version history, deprecations, components that were tried and withdrawn. A showcase page tells you nothing about whether your team can extend it.
• Code and asset ownership, in writing. Who holds what, under which licence, from the day the contract ends.
• Exit terms. Environments, credentials, repositories, documentation — what happens to each on termination, including termination for convenience. Your own risk function will ask you this eventually, because exit planning is an explicit expectation under SS2/21.

Portfolios show the product working. Most of production is the other states. Unfortunately this is also the part nobody prepares a slide for, which is why it sorts suppliers faster than anything else on the list.

• A degraded-state specification. What the interface does when the backend is slow, partially available, or down: what the customer sees, what they can still do, what gets queued. If nobody wrote it down, it was decided by whoever picked up the ticket that week.
• A defect log from one release. What was found before launch, what after, how long each took to close. Ask for the real one (a log with no post-launch defects is a log that wasn’t kept).
• The last incident write-up involving their work. What happened, contributing causes, what changed afterwards. Anyone who has run something at scale has one of these.
• Post-launch support model. Who is on escalation, what hours, what response times, what it costs. Settle it before signature. Afterwards you are negotiating from a weaker position.
• Accessibility and error-state coverage in QA. Whether screen-reader flows and error paths were tested, and by whom. These are the first two things cut when a date moves, and the first two a customer complaint finds.

Scope will change. Undoubtedly some of that will be your doing. What you are testing is whether the supplier has a method for it, or improvises on your budget.

• Change request history from one completed project. How many, what they cost, how long each took to agree. This tells you about their discipline and, separately, about how accurate the original estimate was.
• How an estimate is composed. Which roles, how many days each, what is assumed. A single number cannot be interrogated.
• What moves a range. If they quote a range, ask what takes it from the low end to the high end. No answer means the range is a negotiating position.
• What is excluded. The things clients routinely assume are in scope and aren’t. A supplier who answers quickly has had this argument before and would rather have it now than in month four.
• Their last overrun. What went over, by how much, and who paid for it. The last part is the one that tells you something.

Your risk, security and compliance colleagues will want these eventually. Collecting them during selection costs less than collecting them during onboarding, and nobody is in a hurry yet.

• Certification with validity dates. ISO/IEC 27001 or equivalent: certificate number, issuing body, expiry. A logo on a website is not a certificate.
• The most recent penetration test summary. Scope, date, severity distribution, remediation status. Nobody will share the full report, and nobody should.
• Where data goes. Which jurisdictions, which sub-processors, what leaves the UK or EEA, under what transfer mechanism.
• Accessibility evidence, if the product is consumer-facing. Which WCAG version and conformance level was targeted, how it was tested, by whom. Ask specifically whether testing was automated only. Automated tooling misses screen-reader flows, focus order and semantic structure (which is where the substantive failures are).
• The UK basis. A supplier citing the European Accessibility Act or the Web Accessibility Directive for a UK-only firm has picked up the wrong framework. Your anchors are the Equality Act 2010 and, for consumer outcomes, the FCA’s Consumer Duty.

Don’t send all of it at once. Twenty items at first contact reads as procurement theatre, and you will get a procurement answer back.

What works: on the first call ask for three — the named delivery team with allocation, a decision log from a completed project, and a degraded-state specification. Cheap for a good supplier, expensive for a weak one. Keep the rest for formal evaluation.

Reading a refusal. Undoubtedly some refusals are correct. Confidentiality genuinely prevents sharing an unredacted handover pack, a full penetration test report, or a named incident write-up. A supplier who explains the constraint and offers a redacted version is behaving correctly.

Watch instead for agreement followed by silence. “We can put something together for you” means it doesn’t exist and is about to be written for your benefit. What you’d then be evaluating is their ability to produce a document under deadline.

Two items carry more weight than the rest. The decision log tells you whether the supplier reasons in a way your team can audit after they’ve gone. The degraded-state specification tells you whether they’ve shipped somewhere failure had consequences. Most things on this list can be fixed during an engagement. Those two are either already there or not.

Start with the three. The rest will keep.