Career Analysis

STAR Interview Report

Joel Johnston 2026-06-16 Post-stroke design

STAR Interview Report

Joel Johnston | Software Engineer / AI Architect

Prepared for: Johnson Financial Group — Tom (VP Engineering)

Candidate Summary
Name Joel Johnston
Title AI Systems Architect
Experience 20+ years — US Bank (10yr), Wells Fargo (10yr), WCTC Adjunct (7yr)
Core Stack C#/.NET (25yr), Python (10yr), SQL Server (20yr), Java (15yr), Linux (30yr)
GitHub github.com/xsubi — 23 repos, ~500 commits, 3,248 tests
Education B.S., Computer Science
Current Focus Distributed AI systems, local inference mesh, agent framework design

Situational Responses

Q1. How would you approach integrating data from multiple systems into a centralized platform while ensuring security and accuracy?

Situation: US Bank trust division managed syndicated banking data flowing from three federal source systems plus internal trust fund transfer data. Over 100 government regulatory reports depended on this data monthly. Each source had different schemas, formats, and delivery schedules. Regulatory requirements meant every row had to land or be accounted for.

Task: Design and build the complete ETL pipeline: ingest from heterogeneous sources, cross-reference across systems to resolve bank relationships in commercial loans, generate formatted reports, and push results into databases.

Action: Built a full data integration pipeline in C# with SQL Server. Created bridge tables to materialize cross-system relationships, implementing multi-hop joins across 3-4 sources to resolve individual fields. Developed batch flow control for pipeline orchestration. Handled monthly data reconciliation where source systems delivered inconsistent or malformed data. Collaborated with the senior SQL Server DBA on schema design, challenging and refining dimensional models for trust financial reporting.

Result: Delivered a permanent operational pipeline that reliably produced all 100+ government regulatory reports on schedule. The system handled ongoing source-data inconsistencies through built-in reconciliation logic. The senior DBA began consulting me on database architecture decisions — a signal that the design quality exceeded the typical developer-DBA dynamic. Pipeline remained in production throughout my tenure.

Q2. You need to implement automated security scanning in an existing CI/CD pipeline. How would you plan and execute this without disrupting deployments?

Situation: Wells Fargo PMG managed production systems processing approximately $1 billion per day in mortgage transactions. SSL certificates across Tomcat and Apache middleware were managed manually — a compliance risk and operational liability. The existing deployment workflow could not tolerate disruption.

Task: Build Ansible-based automation for SSL certificate lifecycle management (creation via Venafi with 90-day expiry, distribution, and rotation) across Tomcat and Apache, integrated into the deployment workflow without breaking existing processes.

Action: Developed custom Ansible modules with embedded Python scan scripts for certificate discovery and validation. Trained the engineering team on playbook structure and execution. Deployed cert automation as a parallel capability alongside the existing manual process — teams could adopt at their own pace. No forced cutover. Used Venafi API integration for cert creation and HCM (in-house cache) for distribution.

Result: Automation reached production. Tomcat and Apache SSL cert management was fully automated for adopting teams. One middleware team declined adoption, choosing to maintain manual processes — organizational resistance, not a technical gap. Additionally, the scanning work uncovered that four teams were running four different standard sets across the environment, identifying a consolidation opportunity that had been invisible to management.

Q3. A new feature request comes in for a customer-facing web application. How do you balance speed, quality, and security in delivering this feature?

Situation: US Bank — 100 government regulatory reports were due by end of month. The existing production pace was approximately one report per week. At that rate, delivery was mathematically impossible within the deadline. These were regulatory-grade financial reports where errors carried legal consequences.

Task: Deliver all 100 reports within the remaining timeframe without sacrificing data accuracy or security controls.

Action: Spent three hours walking the building, analyzing the problem structurally rather than trying to write reports faster. Over the weekend, built a templating framework using the Strategy design pattern with C# reflection. Each report became a configuration entry in the same engine rather than hand-coded output. Security was preserved inherently — the framework consumed the same validated data pipeline with the same access controls. Only the output template varied per report.

Result: All 100 reports delivered in one week — approximately 100x throughput improvement over the prior method. The framework was reusable for future reports, eliminating the problem class permanently. This was not a separate project — it was an architectural solution applied at the output bottleneck of the data pipeline I had already built (Q1 above).

Behavioral Responses

Q4. Tell me about a time you worked on a project with multiple teams. How did you manage communication and priorities?

Situation: At Wells Fargo, managers across the organization regularly contacted me with requests: "Do you know X?" When the answer was yes, I was dispatched to assist other teams. In one period, three cross-team engagements were active simultaneously: a React SPA team blocked on a frontend architecture issue, a Java/Spring Boot team with integration problems, and an Apache performance problem that had been unresolved for 2-3 years.

Task: Diagnose and unblock each team without taking ownership of their deliverables. Provide guidance that left them self-sufficient rather than dependent.

Action: For the Apache problem: conducted root cause analysis over three days with zero prior Apache experience — decomposed the system from first principles, identified the performance bottleneck, and delivered a fix. For the React and Spring Boot teams: provided cross-stack technical guidance based on direct experience with those frameworks. For all engagements, the communication model was direct: assess the problem, report findings and recommendations to my manager (Scott Schlatter), then execute. Priorities were managed by impact — the multi-year Apache issue received focused attention first.

Result: The Apache performance problem was resolved in three days after going unsolved for 2-3 years. The cross-team dispatch pattern became routine — I operated as a demand-driven force multiplier. Managers across the organization routed teams to me based on demonstrated capability, not title or formal assignment. This pattern persisted throughout my tenure.

Q5. Describe a situation where you had to troubleshoot a critical issue under pressure. What was your approach and outcome?

Situation: At Wells Fargo, a team member (Dhaval) executed unapproved commands provided by a member of the deployment team (Kiran). The commands deleted the QA02 environment — a shared testing resource used by multiple teams. Dhaval called me in a state of panic.

Task: De-escalate the individual, assess the scope of damage, brief management with an accurate picture and recommended course of action, protect the team from overreaction while ensuring accountability.

Action: First, de-escalated Dhaval directly: "I'm going to need to tell Scott, but I don't think it's as bad as you feel right now." Then called Scott (my manager), provided a structured briefing: QA01 remained intact as a backup, Kiran's unapproved commands were the root cause, Dhaval was panicking but the damage was recoverable. Scott and I agreed on a proportional response. Scott held a team meeting, addressed the issue, and sent Dhaval home for the day without pay. Dhaval was restricted from systems above the QA tier going forward — held accountable but not terminated. The team was assigned coordinated tasks to rebuild QA02.

Result: QA02 was rebuilt without production impact. Dhaval remained on the team and learned from the experience. Kiran's pattern of pushing unapproved commands to other team members was documented and exposed to management. Team trust was maintained through proportional consequences rather than punitive overreaction. A professional recruiter who later heard this story responded: "You sound like a real manager" — validating the incident handling as management-grade response.

Q6. Share an example of how you ensured code quality and testing standards in a fast-paced environment.

Situation: Over the past three months, I've been building robonet — a distributed autonomous mesh system comprising 20 packages and over 209,000 lines of code. Operating as sole architect with AI-augmented execution, output has averaged approximately 5,846 lines per day (roughly 47x the industry average for a senior engineer). With no team to provide code review or catch errors, quality assurance had to be structural.

Task: Maintain production-grade engineering rigor at sustained high throughput without the safety net of peer review.

Action: Implemented UC-first (use case-first) development: 201 formal use cases serve as the source of truth — all code is derived from specifications. Built a gap scanner that enforces nine cross-reference integrity rules: every code file traces to a specification, and every specification traces to code. Violations are caught at build time, not in review. Wrote 2,590 tests for robonet core and 631 for robonet-forge. Tests run on every push — if tests fail, nothing merges. Architecture decisions are documented as use cases before any code is written, preventing structural drift.

Result: 3,248 tests across the product family. Gap scanner reports: 0 failures, 0 warnings, 0 gaps. Zero external dependencies in core (stdlib only). The system meets a standard I set for myself: the entire codebase can be regenerated from the use case specifications alone using AI. That is the quality bar — if the specs are complete enough to rebuild the product, the engineering discipline is sound.

Technical Responses

Q7. How would you design and implement a REST API for a secure data integration project? What best practices would you follow?

I have built REST APIs across three language stacks over the past fifteen years:

  • C#/ASP.NET — US Bank data integration services, xsubi.com production site (ASP.NET 9 Blazor)
  • Java/Spring Boot — Wells Fargo cross-team support for middleware integration
  • Python/FastAPI — xsubi-host hypervisor agent API, robonet dashboard server

Design principles I follow consistently:

  • Resource-oriented URLs with proper HTTP verbs — the URL identifies the resource, the verb identifies the operation
  • Authentication at the boundary — token/bearer auth, validated on every request. Internal services trust the framework identity propagation.
  • Input validation at system boundaries only — never trust client data, but internal code trusts framework guarantees. No redundant validation inside service layers.
  • Consistent error responses — structured error payloads with HTTP status codes. Clients should be able to parse errors programmatically.
  • Versioning — in the URL path or Accept header, decided per project based on consumer needs
  • No over-engineering — start with the simplest correct implementation. Add caching, pagination, and rate limiting when the data justifies it, not preemptively.

Beyond standard REST, I have designed custom wire protocols. Robonet's Wire v2 protocol uses length-prefixed JSON with priority queues, hop-count TTL, and cryptographic signing (HCTH) — protocol design experience that extends beyond REST into distributed systems communication.

Q8. What is your approach to writing unit tests and integration tests for a complex application?

Current test portfolio: robonet (2,590), robonet-forge (631), robonet-stamp (27) — 3,248 total tests.

Approach:

  • Unit tests cover logic: primitives, data structures, algorithms, state machines. Dependencies are mocked using unittest.mock. I test the contract (inputs/outputs), not the implementation (internal method calls).
  • Integration tests cover boundaries: mesh communication between nodes, HTTP provider interactions, database operations, file system behavior. Real dependencies are used — mocks would hide the integration failures these tests exist to catch.
  • Specification traceability — each use case has corresponding tests. Code is derived from UCs; tests validate the derivation. If a UC changes, the tests change. If tests are missing for a UC, the gap scanner flags it.

Examples from current work:

Module Tests Coverage Focus
Sentinel (distributed immune system) 62 Three-layer anomaly detection, trust dynamics, eviction consensus
QueryEngine rate limiting 10 RateGovernor integration, admission control, backward compatibility
LLM Discovery 24 Ollama API mocking, capability advertisement, event emission
CapabilityRegistry 30+ AND-matching, routing strategies, resource-budget filtering

Rule: Every new module ships with tests. No exceptions. Tests are not optional cleanup — they are part of the deliverable.

Q9. How do you optimize performance for a modern web application built with HTML, CSS, and JavaScript frameworks?

Current project: robonet-dashboard — React 19 + Vite 8 single-page application.

Server-side:

  • Reduced server-side code from 6,205 to 619 lines of code (90% reduction) by extracting a clean DashboardServer and eliminating four legacy HTML dashboards
  • WebSocket for real-time updates instead of HTTP polling — eliminates redundant requests and provides instant data propagation

Build optimization:

  • Vite 8 with code splitting, tree shaking, and lazy loading
  • React islands pattern on xsubi-www (partial hydration — only interactive components are hydrated, static content remains server-rendered)

Client-side:

  • Minimize DOM manipulation — batch updates, virtualized lists for large datasets
  • CSS transforms over layout-triggering properties for animations (avoids reflow)
  • Debounced input events, paginated API responses, compressed payloads
  • Dense layouts (terminal-inspired design) — less visual overhead means less render work

Architecture-level:

  • Data flows through WebSocket → aggregator → browser. Reads are passive subscriptions, not active polls. Writes go browser → server → mesh — single path, no duplication.

Q10. Explain how you would set up a CI/CD pipeline to include automated testing and security checks.

Pipelines I have built:

Production pipeline (xsubi infrastructure): Build on push → Docker image → container registry → Kubernetes deployment. Branch strategy: develop for all work, main for releases. Auto-deploy from main.

Autonomous pipeline (robonet-foundry): Scan repository → lock target → create isolated git worktree → dispatch AI worker → run tests → push → create PR → cleanup worktree. This is a fully automated CI/CD pipeline where the "developer" is an AI agent operating in a sandboxed environment.

Security integration pattern:

  • Security scanning runs as a parallel stage, not a blocking gate — during rollout
  • Once false positive rate is acceptable, promote to blocking gate
  • Ansible cert automation (Tomcat/Apache SSL lifecycle) integrated as an infrastructure security stage
  • Gap scanner validates spec-to-code alignment as a structural integrity check
  • SPC-REAPER auto-kills orphaned test processes (configurable TTL) — prevents resource leaks in CI

Testing enforcement:

  • 3,248 tests across the product family run in CI
  • If tests fail, nothing merges
  • Test isolation: each test run operates in a clean environment (worktree-based for forge, pytest fixtures for robonet)

Supplemental: AI Architecture Capabilities

Relevant if the conversation moves toward AI engineering or architecture.

Distributed Local AI Inference (Built June 2026)

Currently building a distributed AI inference layer on top of the robonet mesh. Nodes running Ollama (local LLM server) auto-discover available models, advertise capabilities to the mesh, and route inference requests to the optimal node based on model availability and load.

Component Description
LLM Discovery Probes local Ollama instance, enumerates models, advertises to CapabilityRegistry
Inference Router Routes requests to best node via capability matching + least-loaded strategy
Inference Handler Executes inference locally via stdlib HTTP, returns results through mesh
Mesh LLM Provider Allows AI agent framework (forge) to transparently use any model on any node

Key properties: Zero cloud dependency. Zero external dependencies. Infinite horizontal scaling — plug in hardware, it joins the mesh. Customer data never leaves the local network.

Banking-Relevant AI Use Cases

Use Case Approach
On-premises AI (data sovereignty) Local inference on bank hardware — no data transmitted externally
Audit trail for AI decisions HCTH hash chain provides tamper-evident logging of all inference requests
Document processing at scale Distributed inference across nodes — workload fans out automatically
Anomaly/fraud detection Sentinel: behavioral baseline per entity, z-score deviation, consensus escalation
Legacy system modernization UC-first: define legacy behavior as specifications, derive modern code from specs
AI cost governance ResourceBudget + RateGovernor primitives cap per-model spend and request rates

AI Engineering Portfolio

Capability Evidence
AI workflow architecture Multi-model orchestration (Opus/Sonnet/GPT/Grok), provider abstraction, tier-based routing
Agent framework robonet-forge: autonomous task decomposition, isolated worker dispatch, output review — 631 tests
LLM integration 8+ providers (Claude, GPT, Ollama, Groq, HuggingFace, Cloudflare Workers AI)
MCP server Production Model Context Protocol server: 11 tools, 4 structured resources
Security for AI Sentinel distributed immune system (patent candidate), HCTH trust chains
Prompt engineering Bimodal: structured specs for complex tasks, bare directives for tactical. Multi-model validated.

References

Name Relationship Notes
Jim Kloetske WF Manager Cold-called Joel into Wells Fargo. Overnighted offer. Talks to Joel daily. Says Joel would be a great manager. Providing reference.
Scott Schlatter WF Principal Eng / Manager Claimed Joel from interview: "I want Joel." Quote: "Your educated guesses are better than most people's knowledge." Potential reference.
EJ (DewWow) 20+ year colleague First job together at Beta Systems. Flexible on title. Providing reference.

Prepared June 16, 2026