# TestAbyLity | GenAI Powered testing platform (MVP) learning journey for Testers by Aby Thannikal Joseph

> Generate tests from APIs, UI, and Requirements to supercharge QA teams with AI.

Markdown version of https://testabylity.com/ (a single page; each section below is an anchor on it). Index: https://testabylity.com/llms.txt

## Page metadata

- Canonical URL: https://testabylity.com/
- Social title: TestAbyLity – AI Tools for Testers
- Social description: Generate UI, API, and Requirement-based tests instantly.
- Social image: https://testabylity.com/assets/og-image.jpg (1200×630). Alt text: Test-Abylity: GenAI-Powered Testing Platform, with a screenshot of the sign-in screen
- Keywords: AI for QA, test automation, GenAI testing, REST Assured generator, Selenium to Playwright, QA tools
- Author (structured data): Aby Thannikal Joseph, Test Manager (test manager in UK financial services). Website: https://www.abytjoseph.com/ · LinkedIn: https://www.linkedin.com/in/abytjoseph/ · GitHub: https://github.com/AbyTJoseph · X: https://x.com/aby_t_joseph · Credly: https://www.credly.com/users/aby-thannikal-joseph
- Site banner: Test-Abylity - Solo‑built GenAI test‑automation MVP, created in 2024.

## Page map

| # | Section | Menu label | URL |
| --- | --- | --- | --- |
| 1 | Hero | Home | https://testabylity.com/#hero |
| 2 | Tool capabilities | Tools | https://testabylity.com/#capabilities |
| 3 | See Test‑Abylity in Action | Demo | https://testabylity.com/#video |
| 4 | Product Showcase | Showcase | https://testabylity.com/#product-showcase |
| 5 | Why I Built This | Why I Built It | https://testabylity.com/#why |
| 6 | Architecture at a Glance | Architecture | https://testabylity.com/#architecture |
| 7 | Tech Stack & Tools | Tech Stack | https://testabylity.com/#tech |
| 8 | Build & Learning Journey | Journey | https://testabylity.com/#learning-timeline |
| 9 | Closing Thoughts & Next Steps | Wrap-up | https://testabylity.com/#tradeoffs |
| 10 | Contact | Contact | https://testabylity.com/#contact |

---

## 1. Home

URL: https://testabylity.com/#hero

**TEST-ABYLITY**

GenAI-Powered Testing Platform

Label: ✨ MVP Experiment & Prototype

MVP experiment & prototype built on weekends and late nights over several months exploring GenAI and Large Language Models (LLMs) and future of test automation. Designed, engineered and product-managed an AI test-automation stack.

This prototype helps understand how a single engineer, powered by Generative AI, can achieve an increase in testing throughput.

[Read this page as Markdown](https://testabylity.com/llms-full.txt)

Key themes:

- LLM Function Calling
- Prompt Engineering
- Java Spring + Spring AI
- Multi-LLM Architecture

---

## 2. Tools

URL: https://testabylity.com/#capabilities

Five practical use cases experimented - each built to answer the question : "Could an LLM take this grunt work off my plate?"

| Figure | Label |
| --- | --- |
| 5 | GenAI Tools MVP |
| 2+ | Dual‑vendor LLM Stack |
| 9 | AI Models Evaluated |
| 32 | Learning notes |

### 2.1 Single click API Test Generator

- Tab: API Generator (RestAssured suite in 10s)
- Impact: Drafts tests in seconds
- Highlights: OpenAPI Specs · RestAssured · Positive/Negative & Boundary Tests

Upload any OpenAPI/Swagger file (JSON/YAML) and receive a fully-parameterised **comprehensive RestAssured Test Suite** in under 10 seconds - complete with boundary-condition and field level coverage.

Example shown on the page:

```java
// Generated RestAssured test suite
@Test public void testUsers() {
  given().when().get("/api/users")
  .then().statusCode(200);
}
```

### 2.2 Instant Page-Object Builder

- Tab: Page Objects (Auto-map DOM & metrics)
- Impact: Cuts down boilerplate code
- Highlights: Selenium · Page Object Model (POM) classes · DOM Analysis

Paste raw HTML DOM , and get production-ready **Selenium Page Object Model** classes with locators and page methods generated.

How it works:

1. Paste HTML or URL
2. AI analyzes DOM
3. Generate page objects

### 2.3 Selenium-to-Playwright Migrator

- Tab: Migrator (Selenium → Playwright instantly)
- Impact: Single shot conversion
- Highlights: Java → TS · Minimal Edits

Drop legacy Java Selenium tests and convert to TypeScript Playwright scripts - minimal manual edits required. (MVP had challenges with hallucinations for larger codebases

Example shown on the page:

```java
// Selenium Java
driver.findElement(By.id("login")).click();
```

```typescript
// Playwright TS
await page.locator("#login").click();
```

### 2.4 Requirements Analyzer

- Tab: Req Analyzer (Harden requirements & reduce ambiguity)
- Impact: Highlights ambiguity
- Highlights: Ambiguity Detection · Testability Score · Auto-Suggestions

Upload user stories or requirements documents and receive AI-powered analysis highlighting ambiguities, missing acceptance criteria, and suggestions to improve testability.

How it works:

1. Upload requirements
2. AI analyzes quality
3. Get improvement suggestions

### 2.5 AI Test-Assistant Chat

- Tab: AI Assistant (Answers & insights in <2s)
- Impact: Answers in <2s
- Highlights: Anthropic Claude · Real-time

Open the floating modal from any page and ask *"How to find evolution stage of a Pokemon?"* or *"Show me a list of Pokemons that can fly"*. Chat to query API and product specifications, Streamed answers via Anthropic Claude.

How it works:

1. Ask question
2. AI & RAG analysis
3. Instant results

---

## 3. See Test‑Abylity in Action

URL: https://testabylity.com/#video

Embedded Wistia video: https://fast.wistia.net/embed/iframe/ymz6n35s80 (the video's content is not captured in this file).

---

## 4. Product Showcase

URL: https://testabylity.com/#product-showcase

The sign-in screen plus screen recordings of API test generation and the chat assistant, all from the same local run. Nothing is mocked: this is what the MVP prototype does today.

### 4.1 TestAbylity Login Page

Tags: Primary · Core · Sign-In

Login interface for the GenAI-powered TestAbylity platform with multiple sign-in options. Sign in with Google, GitHub, or plain email—no extra setup. OAuth keeps credentials off the server while you dive straight into the Portal.

Screenshot: https://testabylity.com/assets/Screenshot-1.jpg (1887×989). Alt text: TestAbylity login screen with Google, GitHub and email sign-in

### 4.2 RestAssured Test Generation

Tags: Core · Tool · API Testing

Drop any OpenAPI/Swagger file and receive a fully parameterised RestAssured test suite in seconds. Generates positive, negative, and boundary tests.

Screen recording: https://testabylity.com/assets/TestAbylity-1-APITestGeneration.mp4 (video content not captured)

### 4.3 AI Test Assistant Chat

Tags: Core · Tool · Assistant

Chat with your test requirements: ask details about API specifications, discover API functionalities and endpoints. LLM layer streams answers grounded in real run data.

Screen recording: https://testabylity.com/assets/TestAbylity-2-ChatFunctionality.mp4 (video content not captured)

---

## 5. Why I Built This

URL: https://testabylity.com/#why

Test‑Abylity began as a quick weekend experiment and slowly built out into exploring the capabilities of GenAI in test automation spanning over months of out of the office hours learning to understand --

**"How can LLMs help with testing?"**

That single prompt turned into a prototype where I tried, and sometimes gloriously failed, to stitch together OpenAI, Anthropic, Spring AI, and a React front end. Every hic‑up taught me something new: how to extract clean, JSON‑ready answers via function calling, how to stay model‑agnostic so tomorrow's best model can drop straight in, and how modern front‑end tooling has become surprisingly delightful.

### Learning Goals

- **Master LLM function calling.** Predictable, schema‑safe outputs - no more regex baling wire.
- **Design reusable workflows.** Helps QA teams to focus on higher value tasks.
- **Stay vendor‑neutral.** Today's top model might not be tomorrow's; abstraction layers keep options open.
- **Refresh full‑stack chops.** React hooks, Tailwind flair, Spring Boot APIs, and Spring AI.
- **Eliminate grunt work.** If GenAI can draft a test or parse a log for me, that's a win.

### Tech Stack

- **AI Generation & Prompting:** Anthropic Claude (Sonnet / Opus) as primary, OpenAI GPT as fallback, with domain-tuned custom prompts
- **RAG Orchestrator:** Spring AI's RetrievalAugmentedGenerationModel and VectorStoreRetriever, with custom chunker & metadata enrichment
- **Embeddings & Vector Store:** HuggingFace models with Chroma DB and Jackson/SnakeYAML loaders
- **Frontend:** React 18, React Router v5, etc.
- **Backend:** Spring Boot with REST controllers, TestGenService and POMGenService modules

### Why Launch Now

I finally pushed TestAbyLity.com as a write up of experience building a gGenAI solution as "agentic" AI is here, and waiting longer would only mean reinventing the same wheel later. In a way, I was outpaced by rapid change in the GenAI/LLM space, and the tech stack I built on was more obsolete :) With more advanced solutions appearing every month, there's no better time to ship and learn in public.

---

## 6. Architecture at a Glance

URL: https://testabylity.com/#architecture

| Layer | Technology/Component | Description / Decision Note |
| --- | --- | --- |
| AI Model & Generation Layer | Anthropic Claude (Sonnet/Opus), OpenAI GPT (fallback) | After experimenting with both OpenAI models and Anthropic's Claude models (Sonnet), I stuck with Anthropic for its code quality and structured outputs. The fallback to OpenAI ensured a resilient pipeline even when limits kicked in. |
| Prompt Engineering Layer | Custom Prompts (based on test domain understanding) | I iterated through dozens of prompt versions before landing on one that gave consistently clean JSON responses. Having domain knowledge in testing helped craft the prompts and fine-tune the outputs. It taught me how sensitive LLMs are to tone and phrasing. |
| Authentication Layer | Firebase Auth + Google OAuth 2 | I didn't want to reinvent login or handle authentication myself - Firebase worked in minutes and let me protect routes effortlessly. |
| Presentation Layer | ReactJS | Clean UI was a selling point. Some AI assistance helped (though code-based LLMs weren't that useful at the time). Most components worked well, even for code viewers and charts. As always, styling was the hardest part. |
| Client Logic Layer | React Router, LocalStorage, Form Inputs | I stored results locally so users could revisit without re-processing. This minimized backend load and improved UX. |
| API Layer | Spring Boot REST Controllers | I exposed endpoints for sending front-end data, uploading Swagger, and for the chat service. These became the primary bridge between UI and LLM engine. |
| Business Logic Layer | TestGenService, POMGenService (Spring Components) | This layer gave structure to AI calls - especially when multiple endpoints needed similar orchestration. Separation helped with reusability. |
| Chat Interaction Layer | React modal + Spring `/chat-v1` endpoint | Streaming responses from Anthropic made the chatbot feel alive. Typing indicators and input disabling during load improved UX significantly. |
| File Upload Handling Layer | Spring MultipartConfig + Drag & Drop UI | Simple but essential. I needed to validate files and give instant feedback - many Swagger files failed silently without good checks. |
| Metrics & Visualization Layer | Charts, Tables, Syntax Highlighting | Charts made AI-generated test summaries more tangible. Seeing test counts and coverage visually made it more engaging. |
| RAG Orchestrator Layer | Spring AI `RetrievalAugmentedGenerationModel` | Wiring this up was a turning point - it abstracted the full flow from chunk to Claude with minimal glue code. Switching vector stores or models became trivial. |
| Embedding Model Layer | HuggingFace `bge-small-en` / `bge-base-en-v1.5` | Initially tried OpenAI embeddings, but HuggingFace models gave near-equal performance without needing an API key or GPU - great for quick iteration and local dev. Still learning more. |
| Retriever Layer | Spring AI `VectorStoreRetriever` | Plugging this in gave a quick win - suddenly the chatbot felt aware of Swagger details. |
| Vector Store Layer | Chroma DB (file-based) | I replaced pgvector with Chroma after realizing I didn't need a persistent Postgres instance. Its `.parquet` structure made debugging embeddings easier. This involved more trial and error than expected. |
| Chunking + Metadata Layer | Custom Java logic + DTO Enricher | Chunking by endpoint seemed obvious, but getting metadata tags right (e.g. `auth`, `statusCodes`) was key for retrieval precision. |
| Document Loader Layer | Java (Jackson / SnakeYAML) | Swagger YAMLs were messier than expected. I had to normalize quirks to prevent prompt hallucinations - custom POJOs helped. |
| Analytics & Monitoring Layer | Spring Logging | Enabled logs and dashboards for metrics visibility and debugging. Not flashy, just needed logs when things failed. |
| Build & DevOps Layer | Maven, Spring Boot Starters | Setup was intentionally simple. Maven plugins gave me repeatable builds and helped during local debugging and Dockerization. |

### 6.1 TestAbyLity Architecture

I wanted architectural choices that could survive a real team setting, not just a weekend spike. The diagram below shows how each layer stays swappable and low-ops.

#### Frontend Layer: React

| Component | Parts |
| --- | --- |
| Dashboard | Charts & Analytics; Result Visualization; Session History |
| Upload Forms | Swagger/OpenAPI; Requirements & HTML Input; Selenium Code Import |
| Code Viewer | Syntax Highlighting; Download Features; Diff View |
| AI Chat Modal | Streaming Responses; Interactive Assistant; Floating UI |

**Page Navigation:**

- API Test Generator Page
- Page Object Builder
- Selenium-to-Playwright Converter
- Requirements Analyzer
- Settings & Profile

- **Sidebar:** Persistent navigation with collapsible sections
- **Routing:** React Router v5 with protected routes
- **State:** Context API + LocalStorage Persistence

#### Backend Layer: Spring Boot 3.x + Maven

REST API Endpoints:

```http
POST /generate-api-tests
POST /pom-generator-v1
POST /ui-scripts-migrator-v1
POST /req-analyzer-v1
POST /chat-v1
GET /health
```

| Component | Parts |
| --- | --- |
| File Processing | Swagger/OpenAPI Parser; HTML DOM Analysis; Requirements Parser |
| Code Generation | RestAssured Tests; POM Classes; Playwright Migration |
| Spring AI Integration | ChatClient Interface; RetrievalAugmentedGenerationModel; Structured Prompting Engine |

#### Authentication: Firebase + Google SSO

| Component | Parts |
| --- | --- |
| Firebase Auth | Google Sign-In; JWT Token Management |
| Frontend Integration | AuthContext Provider; Protected Routes; Session Management |

#### AI Services: Multi-Model LLM Architecture

| Component | Parts |
| --- | --- |
| Anthropic Claude | Primary AI Engine (Sonnet/Opus); Test Case Generation; POM Class Creation; Requirements Analysis |
| OpenAI GPT | Fallback Option (GPT-3.5); Chat Responses; Code Analysis; Resilient Pipeline |
| Vector Embeddings | HuggingFace bge-small-en; bge-base-en-v1.5; Chroma DB Storage |
| Prompt Engineering | Structured System Prompts; JSON Schema Validation; Function Calling; Chain-of-Thought Processing |

### 6.2 Core Feature Data Flows

**API Test Generation**

1. Upload Swagger/OpenAPI File
2. Parse Schema and Endpoints
3. AI Analysis for Test Scenarios
4. Generate RestAssured Code
5. Display Results with Charts

**POM Generation**

1. Input HTML Snippet
2. DOM Element Analysis
3. AI Generate POM Classes
4. Java/TypeScript Output
5. Download Generated Code

**Requirements Analyzer**

1. Upload Requirements Document
2. Ambiguity Detection
3. Testability Analysis
4. Generate Improvement Suggestions
5. Provide Testability Score

**Selenium Converter**

1. Input Selenium Code
2. Syntax Analysis
3. AI Transform Logic
4. Playwright Code Output
5. Interactive Stepper UI

**AI Chat Assistant**

1. User Query Input
2. RAG Context Retrieval
3. AI Response Generation
4. Stream Response to UI
5. Interactive Chat Modal

---

## 7. Tech Stack & Tools

URL: https://testabylity.com/#tech

LLM-agnostic architecture with swappable components

| Area | Summary | Item | Role |
| --- | --- | --- | --- |
| AI Generation & Prompting | Dual-vendor LLMs with domain-tuned prompts for robust test generation. | Anthropic Claude (Sonnet / Opus) | Primary |
| | | OpenAI GPT (fallback) | Fallback |
| | | Domain-tuned custom prompts | Core |
| RAG Orchestrator | Advanced retrieval system orchestrating context-aware AI responses. | Spring AI RetrievalAugmentedGenerationModel | Core |
| | | Spring AI VectorStoreRetriever | Core |
| | | Custom chunker & metadata enricher | Tool |
| Embeddings & Vector Store | High-performance vector storage and embedding models for semantic search. | HuggingFace bge-small-en / bge-base-en-v1.5 | AI |
| | | Chroma DB (file-based .parquet) | Data |
| | | Document loaders (Jackson / SnakeYAML) | Tool |
| Frontend UI | Modern React-based interface with rich data visualization. | React 18 | Core |
| | | React Router v5 | Tool |
| | | Charts, tables & syntax highlighting | UI |
| Backend Services | Spring Boot services powering core logic and API endpoints. | Spring Boot REST Controllers | Core |
| | | TestGenService (Spring) | Core |
| | | POMGenService (Spring) | Core |
| Chat Interaction | Interactive chat interface with real-time streaming and UX enhancements. | React modal chat widget | UI |
| | | Streaming /chat-v1 endpoint | Core |
| | | Typing & loading UX hints | UI |
| Client State & Auth | Secure authentication and client-side state management. | Firebase Auth | Core |
| | | Google OAuth 2 | OAuth |
| | | LocalStorage caching | Tool |
| Build & Ops | Streamlined build process and operational monitoring. | Maven (Spring Boot starters) | Tool |
| | | Dockerized worker images | DevOps |
| | | Spring logging & dashboards | Monitoring |

---

## 8. Build & Learning Journey

URL: https://testabylity.com/#learning-timeline

Breadcrumbs of code spikes, bug hunts & aha-moments.

| Figure | Label |
| --- | --- |
| 32 | Learning Points |
| 40+ | Technologies |
| 9 | AI Models |
| 100% | Hands-On |

### Phase 1: Foundation & Core Technologies

#### Step 1. Spring Boot Refresher

- Stage: Foundation
- Outcome: ✅ Service + Tests
- Technologies: Spring Boot 3.x · REST Service · Testing
- Insights: Knowledge Refresh

I kicked off by rebuilding a simple REST service just to shake off the rust - Spring Boot had changed more than I remembered. Reacquainting myself with annotations and tricks helped me get back on track. Watching live reloads work again felt oddly satisfying.

#### Step 2. REST API Foundation

- Stage: Service Layer
- Outcome: 🔗 API Bridge
- Technologies: Spring Boot REST · CORS · Multipart Config
- Insights: LLM-UI Bridge Setup

Getting the backend to talk to the frontend wasn't as smooth as I expected - CORS issues, multipart limits, and MIME headers all needed tuning. I used Postman and browser dev tools in loops until errors stopped. Fixing this gave me a clearer picture of what real-world API bridging involves.

#### Step 3. Service Architecture

- Stage: Logic Layer
- Outcome: 🏗️ Service Design
- Technologies: TestGenService · POMGenService · Spring Components
- Insights: Clean Orchestration · Reusability

When wiring up AI routes, I quickly saw things getting messy. Breaking them out into TestGenService and POMGenService gave clarity. I didn't expect how reusable these would become. Debugging was a easier after separating the logic.

### Phase 2: AI Model Exploration & Selection

#### Step 4. OpenAI Experiments

- Stage: Benchmarking
- Outcome: 📊 Performance Analysis
- Technologies: text-davinci-003 · GPT-3.5-turbo · GPT-4
- Insights: Cost vs Quality

I ran dozens of prompts through OpenAI's models - some evenings were just latency tests and JSON parse failures. I created a local sheet comparing response length, price, and formatting reliability. It made the tradeoffs really concrete.

#### Step 5. Spring AI Deep‑Dive

- Stage: Discovery Phase
- Outcome: 🎯 Key Discovery
- Technologies: Spring AI · Function Calls · JSON Output
- Insights: Game Changer

I cloned their samples and traced every piece - function calls, prompt formatting, fallback logic. That's when I stumbled on the Spring AI, which later became critical. It felt like finding a hidden lever.

#### Step 6. Anthropic Claude Integration

- Stage: Model Selection
- Outcome: 🏆 Winner Selected
- Technologies: Claude Sonnet 3.7 · Java Tests
- Insights: 15% Shorter Code · 40% Cheaper

I dropped Claude 3 Sonnet into the same prompts I used with GPT. To my surprise, Claude's output was shorter, cheaper, and just as clean. I ran diff comparisons between them to confirm it wasn't a luck.

#### Step 7. Multi‑Model Evaluations

- Stage: Comprehensive Test
- Outcome: 🔬 Model Showdown
- Technologies: Claude 3 Opus, Claude Sonnet 3.7, Claude Haiku 3 (Testing) · GPT · OpenAI Fallback
- Insights: Quality‑Resilient Pipeline

This was the fun part - prompt packs, retries, OpenAI as fallback. I intentionally pushed rate limits to simulate worst-case. Watching the pipeline hold up taught me the real value of fallbacks and model orchestration.

#### Step 8. AI Model & Generation Layer

- Stage: GenAI Core Layer
- Outcome: 🔄 Model Orchestration
- Technologies: Claude · OpenAI · Model Switching
- Insights: Quality‑Resilient Pipeline

Final setup had Claude (Sonnet/Opus) for day-to-day, and OpenAI as a safety net. When Claude hiccupped, OpenAI took over. I tested fallback by toggling API keys mid-session. It worked - and gave me peace of mind.

### Phase 3: Prompt Engineering & Quality

#### Step 9. Prompt‑Engineering Basics

- Stage: Pattern Discovery
- Outcome: 📈 Documented Gains
- Technologies: Zero-shot · Few-shot · Chain-of-Thought
- Insights: Increased prompt accuracy

I tried zero-shot first - results were all over the place. Few-shot helped, but chain-of-thought unlocked tricky cases. I kept a prompt history log just to track improvements. Coverage jumped and errors dropped.

#### Step 10. Prompt Benchmark Script

- Stage: Tooling Phase
- Outcome: 🔧 Benchmarking Tool
- Technologies: Python CLI · Latency Tracking · Token Stats · JSON Validation
- Insights: Unified Metrics

Got tired of eyeballing performance. I wrote a CLI script to dump latency, token count, and JSON parse status into CSV. Every test run now leaves a footprint - makes A/B testing trivial.

#### Step 11. Prompt Fine‑Tuning

- Stage: Quality Improvement
- Outcome: 📉 Major Error Reduction
- Technologies: JSON Schema · System Prompts
- Insights: 28% → 3% Error Rate · Parse Reliability

At one point, parse errors hit 28% and most of my prompts included a constant reminder "please respond only in JSON, please", I added a strict JSON schema to the system prompt and re-ran the same test set - errors dropped to 3%. I felt like a backend QA engineer at that moment.

#### Step 12. Prompt Engineering Layer

- Stage: Prompt Optimization
- Outcome: 🎯 Domain Expertise
- Technologies: Custom Prompts · Testing Domain Knowledge
- Insights: Tuning · Domain Specific

This phase was gritty. I went through 20+ prompt iterations, hand-checking each one. Once I started using testing-specific tone and consistent formatting, JSON stopped breaking. Domain knowledge really helped here.

#### Step 13. Guardrails & Schema Validation

- Stage: Quality Assurance
- Outcome: 🛡️ Error Prevention
- Technologies: Schema Validation · Re-ask Loops
- Insights: Zero Silent Errors · Robust Validation

Silent failures haunted me early on. Wrapping outputs in schema validations and adding re-adding prompts reduced errors. The structured output wasn't often structured, and this minimised the issue as possible

### Phase 4: Frontend Development

#### Step 14. Frontend Build & Troubleshooting

- Stage: Frontend Phase
- Outcome: 🖼️ UI Issues Resolved
- Technologies: ReactJS
- Insights: Style Conflicts Fixed · Lazy Loading

The front end template and the libraries used didn't play nice at first. Some styles overrode others, and lazy loading failed silently. I had to inspect bundles and isolate imports - remided me how difficult UI development can be

#### Step 15. Frontend Dashboard Build

- Stage: UI Design Phase
- Outcome: 🎨 UI Masterclass
- Technologies: ReactJS · Charts · Drag‑and‑Drop
- Insights: Visual Clarity · UX Lessons

Building the dashboard was a UI masterclass - drag-and-drop uploads, syntax highlighting, animated modals. I ran dozens of tests just to make one button feel responsive. The polish came from pain.

#### Step 16. Authentication Layer

- Stage: User Access Control
- Outcome: 🔒 Security Layer
- Technologies: Firebase · OAuth 2
- Insights: Quick Auth Setup

I plugged in Firebase + Google Auth and was stunned it just worked. Securing routes in under 60 minutes made me wonder why I ever tried doing this manually before.

#### Step 17. Client State Management

- Stage: UX Enhancement
- Outcome: 💾 State Management
- Technologies: React Router · LocalStorage · State Persistence
- Insights: State Handling · Session Continuity

Using localStorage + React Router let users pick up where they left off. I tested persistence across tabs and refreshes. This small addition made the whole UX feel premium.

### Phase 5: Full-Stack Integration

#### Step 18. Backend ↔ Frontend Wiring

- Stage: Integration Phase
- Outcome: 🔗 Full Stack Connected
- Technologies: Axios · CORS · Multipart · MIME Headers
- Insights: Integration Complete

Cleared CORS gremlins, raised the 413 limit and fixed MIME headers - backend and UI finally shook hands. I tested with curl, browser uploads, and large Swagger files until it clicked. Felt like debugging plumbing leaks.

#### Step 19. File Upload Pipeline

- Stage: Input Handling
- Outcome: 📁 File Processing
- Technologies: Multipart Upload · File Validation · Toast Notifications
- Insights: UX Validation · Error Prevention

MultipartConfig helped trap malformed Swagger files before they silently broke everything. I added toast notifications after misclicks left me confused. It was hands-on validation tuning with lots of trial and error.

#### Step 20. Chat Interface Integration

- Stage: Chat Feature Build
- Outcome: 💬 Chat Experience
- Technologies: React Modal · Streaming Responses · Spring Chat Endpoint · Typing Indicators
- Insights: Real-time Conversational UX

Streaming Claude replies into a floating modal looked simple - but debugging typing indicators and reset flow was more work than expected. It made me think like a bot interaction designer.

#### Step 21. Results Dashboard

- Stage: Results Visualization
- Outcome: 📊 Data Display
- Technologies: Charts · Syntax Highlighting · Code Display · Metrics Dashboard
- Insights: Data Visualization · Code Presentation

I split POM and API results into separate dashboards using localStorage as a poor man's state store. Chart rendering broke twice on malformed data - fixing that gave me a new respect for UI error boundaries.

### Phase 6: RAG & Vector Search Implementation

#### Step 22. Document Loader Layer

- Stage: Loader Setup
- Outcome: 📄 Document Processing
- Technologies: Jackson · SnakeYAML
- Insights: Cleaner Input Pipeline

Swagger YAMLs were far messier than I expected. Parsing failed on nested examples, inline enums, and missing fields. I cleaned them with Jackson and custom POJOs - felt like prepping data for a finicky chef.

#### Step 23. Chunking + Metadata Layer

- Stage: Preprocessing Layer
- Outcome: 🔍 Content Analysis
- Technologies: Java Chunker · Metadata Enricher
- Insights: Precision Targeting

I wrote a Java chunker that tagged endpoints with verbs, auth status, and status codes. Debugged it using real Swagger samples and compared what was captured vs missed. Metadata precision became my obsession.

#### Step 24. Embedding Model Layer

- Stage: Vectorization
- Outcome: 🧠 Local Embeddings
- Technologies: HuggingFace · bge-small-en
- Insights: Local Dev · Cost‑free Iteration

Swapped OpenAI embeddings with HuggingFace's bge-small-en. I ran tests side-by-side on the same chunks and measured cosine similarity in local notebooks - no API keys, no stress.

#### Step 25. Vector Storage Optimization

- Stage: Database Evaluation
- Outcome: 🚀 Performance Winner
- Technologies: PGVector · Chroma DB · ElasticSearch
- Insights: File-based · Lightweight · 2× Faster

Compared PGVector, Chroma and ElasticSearch. File-based Chroma won - it was easier to inspect raw vectors and debug weird retrieval issues. Rebuilding the store was 2× faster too.

#### Step 26. Retriever Layer

- Stage: Retriever Config
- Outcome: 🔍 Knowledge Retrieval
- Technologies: Spring AI Retriever
- Insights: Retrieval Accuracy

I wired up Spring AI's VectorStoreRetriever and immediately tested Swagger Q&A queries. Seeing the bot return endpoint names and auth rules from memory was a turning point - it felt aware.

#### Step 27. RAG Orchestrator Layer

- Stage: Retrieval Pipeline
- Outcome: 🔄 RAG Orchestration
- Technologies: Spring AI RAG · Claude
- Insights: RAG Simplified

I was stunned how clean Spring AI's RAG model API was - just passed chunks and prompts, and it handled routing. I toggled Claude ↔ OpenAI in 3 lines. This changed how I viewed multi-model wiring.

### Phase 7: Advanced Research & Experiments

#### Step 28. Code‑Conversion Spike

- Stage: Research Spike
- Outcome: 🧪 Migration Strategy
- Technologies: Selenium · Playwright · Code Migration
- Insights: Edge Cases Documented · POC Complete

I prototyped a tool to migrate Selenium to Playwright tests. Ran it on my old UI suite - some locators broke, some DOM assumptions didn't translate. Every failure gave ideas for smarter prompt scaffolding.

#### Step 29. Agent Pattern Prototype

- Stage: Architecture Research
- Outcome: ⚡ Agent Prototype
- Technologies: Spring AI · Tool-call Chaining · Agent Pattern
- Insights: Promising Results · Future Architecture

I chained together tool-calls in Spring AI to go from spec → test → cleanup. Half of it broke, but when it didn't - it was magical. It felt like watching agents reason. Definitely unfinished, but full of promise.

### Phase 8: Documentation & Operations

#### Step 30. Knowledge Base Creation

- Stage: Documentation
- Outcome: 📚 Knowledge Base
- Technologies: Regex Extractors · System Templates · Model Comparison · Latency Metrics · Cost Analysis · Context Limits
- Insights: 50% Faster Onboarding · Standardized Patterns

I rolled all the regex extractors and prompt patterns into a shared cookbook. The latency vs context vs cost matrix saved me mid-debug more than once. Onboarding now takes 50% less time - and fewer Slack pings.

#### Step 31. Analytics & Monitoring Layer

- Stage: Debug Phase
- Outcome: 📊 Monitoring Setup
- Technologies: Spring Boot Logging
- Insights: Visibility with Simplicity

Added just enough logging in Spring Boot to flag when things silently failed. Watching request cycles and payload dumps during dev made debugging much less guesswork.

#### Step 32. Build & DevOps Layer

- Stage: Infra Layer
- Outcome: 🏗️ Build System
- Technologies: Maven · Spring Starters
- Insights: Repeatable Build Flow

I kept the build stack minimal - Maven, Spring Starters, and a Dockerfile. Ran dry builds repeatedly until it stopped throwing surprises. Simple, predictable, ready to ship (but I paused here :) )

---

## 9. Closing Thoughts & Next Steps

URL: https://testabylity.com/#tradeoffs

Since the time I built this, the tooling landscape has also shifted - many developer tools and IDEs now offer built-in code generation features that simply didn't exist back then.

I'm writing this now because GenAI and LLM technology have rapidly evolved. This project was a way to explore the space hands-on - and now that it's built, I'm treating this as a thoughtful wrap-up, not a SaaS launch. Test‑Abylity served its purpose: it let me experiment, learn, and contribute something useful to the conversation around GenAI in testing.

From here, I'm excited to move on and explore new, innovative solutions that take QA even further.

Exploring Agentic systems and multi-agent systems, model context protocols and others; maybe this project needs a complete revamp with a different lens now 😊

---

## 10. Contact

URL: https://testabylity.com/#contact

**Aby Thannikal Joseph**

Got thoughts or seen a similar challenge? Drop me a note on LinkedIn -- always keen to chat testing & AI.

- Message me on LinkedIn: https://www.linkedin.com/in/abytjoseph/ (the "Chat with Me" button links here too)
- More of my writing on testing and AI: abytjoseph.com (https://www.abytjoseph.com/)
- Profile photo: https://testabylity.com/assets/avatar-240.jpg (alt text: Profile Image; initials "AJ" are shown if it fails to load)
- Watch the Demo: https://testabylity.com/#video

Not captured: the clap button's visitor count and the Disqus comment thread, which are loaded by JavaScript.

Footer: ⚙️ Powered by Anthropic Claude (LLM-agnostic backend). © 2024 TestAbyLity. All Rights Reserved. Built by Aby Thannikal Joseph

---

## Interface text

- Banner at the top of the page: "Are you an AI agent? There's a Markdown-friendly version of this page for you here." with a "View Markdown" link to this file (https://testabylity.com/llms-full.txt).
- Menu (side nav and mobile menu), after the section links: "Markdown version", a link to this file (https://testabylity.com/llms-full.txt).
- Section toggles: "Read the full journey" / "Show less" (Why I Built This); "Show all 32 learning steps" / "Hide learning steps" (Build & Learning Journey).
- Video fallback text: "Your browser does not support the video tag."