VIREN/SHAH
← Back to the portfolio

PROJECT CASE STUDIES

How I approached the work and what changed.

Each case study starts with the problem and follows the work through its production result. I also explain where experience from another part of my background changed a decision.

Filter by discipline

Showing all 17 projects

BlackBoiler / applied AI platform

Extending and hardening BlackBoiler's multi-agent platform

  • AI / ML
  • Platform
  • Security
  • Product engineering

I extended BlackBoiler’s existing multi-agent platform to improve contract editing and make legal playbook models easier for users to manage. My work also covered model support, tenant isolation, deployment, and operational visibility.

Starting from the existing system

The original implementation already routed contract editing, playbook management, clause search, analytics, and rule diagnosis to specialized agents. I extended that system as the product added a web assistant, administrative functions, analytics, and new execution paths.

Editing and playbook management

I built the web assistant application and added tools for subscriptions, accounts, playbooks, clauses, and analytics. The playbook workflows aligned contract sentences more accurately with the rules and gave users the ability to create and modify their own legal playbook models. Before this work, playbooks had to be managed by people with specialized domain knowledge. I also introduced a shared model factory so the agents could use Gemini or Bedrock without managing the provider in every agent.

Tenant isolation and access control

I added bearer-token authentication, centralized the scope checks used by the agents, and strengthened request isolation. I also added tests covering concurrent tenant activity.

Reliability and operation

I added retries for transient Gemini failures, fixed delegation so control returned to the web root agent, and tagged MLflow spend with the resolved tenant. I also added container builds with SBOM and provenance attestations.

Impact

The changes made contract editing faster and more accurate by improving how sentences align with playbook rules. For the first time, users could create and modify their own legal playbook models without the specialized domain knowledge the earlier process required.

Technologies & skills

  • Python
  • Google ADK
  • Gemini
  • AWS Bedrock
  • LiteLLM
  • MCP
  • FastAPI
  • SSE
  • contextvars
  • MLflow
  • JWT authentication
  • SBOM
  • pytest

BlackBoiler / production AI

Automatic threshold selection for legal playbook rules

  • AI / ML
  • Platform
  • R&D

I replaced manual similarity-threshold estimates with a production service that evaluates each rule’s training data and recommends thresholds automatically.

The problem

Contract analysts inspected training examples and estimated the similarity values that decide whether a rule should fire or whether the proposed language is already present. The process depended on judgment, took time, and could vary from one rule to the next.

The analysis

I compared a broad set of techniques for accuracy and runtime. Based on those results, I combined gap detection, kernel density estimation, Gaussian mixtures with BIC, coefficient of variation, and a bimodality coefficient into a consensus recommendation with low, middle, and high strategies.

More than one number

The same analysis detects weak clusters that may need splitting, IQR outliers, and near-duplicate examples. The duplicate check combines trigram Jaccard similarity with SequenceMatcher and groups related examples together instead of returning a noisy list of pairs.

From prototype to production

I took the work from a Python CLI and library to an EKS service with asynchronous jobs, Helm deployment, persistence, parallel analysis, tests, and an SBOM-attested container. It now recalculates thresholds automatically after a rule’s training data is enhanced.

Impact

The service replaced manual threshold estimates in the rule-enhancement pipeline. Analysts can create playbooks faster, while rules use more consistent thresholds and weak training clusters are flagged before they affect results.

Technologies & skills

  • Python
  • KDE
  • GMM/BIC
  • Statistical analysis
  • Similarity metrics
  • FastAPI
  • EKS
  • Helm
  • Container security
  • SBOM
  • pytest

BlackBoiler / LLM evaluation

Judges and guardrails for generated contract edits

  • AI / ML
  • Security
  • Product engineering

I helped build and harden the evaluation layer that decides whether to keep a generated edit, where new language belongs, and whether similar language is already present.

The evaluation problem

A generated edit may read well but be wrong for the rule or contract. Separate judges evaluate retention, placement, duplication, intent, and inconsequential changes. They use the rule’s human-labeled examples because raw document diffs can be misleading.

Making model output dependable

I consolidated several slightly different JSON cleanup paths into shared normalization code that handles code fences, trailing commas, object-literal syntax, and control characters. This reduced inconsistent behavior when models returned almost-valid structured output.

Concurrency and observability

When evaluation calls moved into a thread pool, I made shared reporting and timing state safe under concurrency and preserved tenant context across worker boundaries. That restored accurate tracing and per-tenant cost attribution during parallel execution.

Guardrails in production

I worked on Bedrock guardrails for the points where generated text reaches the product and moved from a separate chunked pre-check to native model-call integration. I also corrected the response interpretation that had caused normal calls to be reported as blocked.

Impact

The evaluation layer catches poor, misplaced, or duplicate edits before they reach users. Shared parsing, guardrails, and concurrency fixes made the results more consistent and restored accurate tracing and cost attribution.

Technologies & skills

  • Python
  • AWS Bedrock
  • Claude
  • LLM evaluation
  • Guardrails
  • Pydantic
  • MongoDB
  • Concurrency
  • Context propagation
  • MLflow
  • pytest

BlackBoiler / deployment engineering

Turning tenant deployment scripts into an internal product

  • Platform
  • Security
  • Product engineering

The original deployment approach was a set of scripts and infrastructure-as-code that had grown organically with the SaaS platform. It touched more than two dozen cloud services and product components, and deploying a tenant required an infrastructure specialist who understood how those pieces fit together.

How it started

The scripts provisioned tenant-specific storage, encryption keys, DNS, email, networking, identity, data services, and Kubernetes resources. They automated much of the work, but running them still required infrastructure knowledge and offered little help with validation, progress, or recovery.

The service

I turned that deployment process into a React and FastAPI application with an 18-step workflow, from preflight checks through Helm and ArgoCD rollout. Deployments take 10–20 minutes, so they run as polled background jobs. Dry runs catch problems early, and destructive actions require typed confirmation.

Identity at scale

A shared OAuth client had a fixed redirect limit that would eventually block growth. I automated a per-tenant SAML application and Cognito identity-provider setup instead, then fixed an authentication issue in the login flow involving several identity layers.

Safer operations

The service uses in-cluster AWS identity, so it does not store operator credentials or require local Kubernetes tooling. Dependency and container hardening reduced critical and high-severity findings.

Impact

Customer Success gained a repeatable way to deploy and troubleshoot client environments directly. Preflight checks and recoverable background work reduced deployment risk and operator effort.

Technologies & skills

  • Python
  • FastAPI
  • React
  • TypeScript
  • AWS Cognito
  • EFS
  • S3
  • DocumentDB
  • RabbitMQ
  • Helm
  • ArgoCD
  • IRSA
  • SAML/OAuth/PKCE

BlackBoiler / distributed workflows

Automating signup and tenant provisioning

  • Platform
  • Security
  • Product engineering
  • Distributed systems

I built a self-service signup and provisioning flow that coordinates identity, infrastructure, and application setup across Cloudflare and AWS.

The system boundary

The flow spans a Next.js interface, Cloudflare Workers and Durable Objects, AWS Cognito and Lambda, and backend provisioning services. It also includes bot protection and machine-to-machine authentication between the edge and AWS-hosted services.

At-least-once delivery

Identity confirmation can emit the same event more than once. I assigned each user a Durable Object so its single-threaded execution acts as a distributed lock, preventing two simultaneous tenant builds without relying on timing or a fragile in-memory flag.

Async failure modes

I traced lost fire-and-forget Lambda work, a race between credentials email and tenant readiness, and a background task that could outlive the Worker request. The final design confirms the handoff quickly, then continues provisioning on the server with its own retry logic.

Resumable provisioning

The backend persists every provisioning step and checks completed work before resuming. I also replaced thousands of individual training-document writes with a bulk background copy, reducing request latency and avoiding manual cleanup after partial failures.

Impact

New customers can move from signup to a provisioned tenant through an automated workflow. The service coordinates the systems involved and can resume work safely, reducing manual setup and cleanup.

Technologies & skills

  • Next.js
  • Cloudflare Workers
  • Durable Objects
  • AWS Cognito
  • AWS Lambda
  • Python
  • FastAPI
  • MongoDB
  • Idempotency
  • Distributed locking
  • M2M authentication
  • Vitest

BlackBoiler / platform operations

Making multi-cluster operations safer and less expensive

  • Platform
  • Security
  • Distributed systems

I improved the reliability, cost, observability, and access controls of the Kubernetes environments that ran our production and staging systems.

Service-mesh upgrade

I planned a six-hop Linkerd upgrade from an end-of-life release. The sequence had to stay within Linkerd’s supported control-plane and data-plane version skew, and I moved a Gateway API ownership change to the one hop where it could be made safely. I checked for pod disruption after every step.

Finding the source of the cost increase

A telemetry pipeline jumped from about $6–7 per day to roughly $140. I traced the increase to duplicate per-node collection and metric-name collisions, changed the collector topology, and moved alerting to self-hosted Prometheus and Alertmanager. Cost fell about 98%.

An operational interface

I built a multi-cluster dashboard for nodes, pods, logs, and resource use. It preserves node history in DocumentDB, uses a separate aggregate query so log histograms are not distorted by result limits, and reaches Prometheus safely through the Kubernetes API proxy.

Security and response

I replaced broad cluster-admin access with the single permission the dashboard actually needed and added alert enrichment that includes the relevant pods, nodes, and log lines. Operators can act from the alert itself instead of reconstructing the failure across several consoles.

Impact

The changes made production easier to operate and less expensive. They reduced one telemetry pipeline’s cost by about 98%, tightened cluster access, and gave operators faster access to the context needed to respond to alerts.

Technologies & skills

  • AWS EKS
  • Kubernetes
  • Linkerd
  • Gateway API
  • Prometheus
  • Alertmanager
  • CloudWatch Logs Insights
  • DocumentDB
  • IAM/RBAC
  • Cost analysis
  • React
  • Node.js

BlackBoiler / production reliability

Event-driven document processing without costly rework

  • Platform
  • Distributed systems

I changed the compute model for document-processing jobs after Spot interruptions began causing too much repeated work.

The workload

KEDA created Kubernetes jobs as documents entered the processing queue. Demand-driven scaling matched the product well, but the jobs were long enough that frequent Spot-instance churn could interrupt work close to completion.

The operational failure

Interrupted documents returned to the queue and had to be processed again. That increased latency, made throughput less predictable, and consumed compute without producing a completed result. The cheapest hourly instance was not the cheapest completed job.

The decision

Dedicated on-demand EC2 capacity was too costly. I moved the affected jobs to AWS Fargate, preserving KEDA’s event-driven behavior while avoiding dependence on interruptible nodes. I also addressed duplicate pickup during slower Fargate pod starts and a CSI volume incompatibility.

What changed

The new design reduced interruptions and repeated processing while keeping capacity tied to queue demand. I reached that decision by looking at application behavior, scheduler timing, storage, reliability, and the cost of a completed document together.

Impact

Document processing became more predictable and wasted less compute on jobs interrupted near completion. Queue-driven scaling kept capacity tied to demand, and Fargate avoided the cost of dedicated on-demand nodes.

Technologies & skills

  • Kubernetes
  • KEDA
  • AWS Fargate
  • Amazon EC2 Spot
  • Event-driven jobs
  • Queue processing
  • CSI storage
  • Reliability engineering
  • Cost modeling

BlackBoiler / frontend platform

Bringing the web interface and Word add-in onto one frontend platform

  • Platform
  • Product engineering

I brought BlackBoiler’s web interface and Word add-in into a pnpm monorepo and extracted a shared design system for both interfaces.

The starting point

The web interface and Word add-in lived in separate projects and had begun to diverge in components, dependencies, and build practices. Sharing changes meant repeating work or maintaining slightly different versions of the same component.

The monorepo and design system

I moved the applications into a single pnpm workspace and extracted rocket-ui, a shared component package built on Chakra UI. I led the migration away from Material UI, including common selection, upload, dialog, and form controls.

Checking the migration visually

I built a small visual-regression tool to compare the old and new interfaces where normal unit tests could not catch layout differences. I also replaced hardcoded colors with semantic tokens so shared components behaved correctly in dark mode.

A foundation for later work

I set up package-specific type-check and test jobs in GitHub Actions, with concurrency and trigger rules that avoided unnecessary runs. The monorepo became the base for the authentication, settings, and AI-assistant work that followed.

Impact

The monorepo and shared component library reduced duplicated work across the web interface and Word add-in. They also provided a common base for later authentication, settings, and assistant features, with CI and visual checks to protect changes.

Technologies & skills

  • React
  • TypeScript
  • pnpm
  • Monorepos
  • Chakra UI
  • Design systems
  • GitHub Actions
  • Visual regression
  • Dark mode
  • Frontend migration

BlackBoiler / Word add-in authentication

Rebuilding authentication for the Word add-in

  • Platform
  • Security
  • Product engineering

I rebuilt the Word add-in’s authentication around Cognito SRP and Auth0 PKCE, fixing recurring login windows, cross-tab logout races, and token storage that did not distinguish between tenants.

An unusual browser environment

Office add-ins run inside a constrained webview, where a normal redirect-based login can create nested dialogs and behave differently from a browser tab. The old flow also discarded sessions as soon as their access tokens expired.

Two authentication paths

I built an email-first Cognito flow using the Secure Remote Password protocol, so the password is never sent directly, and a separate Auth0 flow using PKCE for OAuth dialog logins. Silent refresh restores an existing session when the add-in opens.

Tenant-safe token storage

I isolated authentication state by tenant and added a migration path for existing sessions. This resolved the login and logout conflicts users encountered across tabs and tenant environments.

Regression protection

The tests cover tenant isolation, concurrent refresh-token requests, and OAuth popup state used to prevent request forgery. I also added a dedicated CI check for the authentication storage rules.

Impact

Users could return to the add-in without repeated login prompts, and sessions no longer collided across tenants and tabs. The tests made those guarantees part of the release process.

Technologies & skills

  • Office.js
  • React
  • TypeScript
  • AWS Cognito
  • Auth0
  • SRP
  • OAuth PKCE
  • Token isolation
  • Session renewal
  • Authentication testing

BlackBoiler / multi-cloud infrastructure

Reproducing the SaaS platform in GCP in six weeks

  • Platform
  • Security
  • Product engineering
  • Distributed systems

A client required GCP hosting, while BlackBoiler ran in AWS. I reproduced the platform on GCP and delivered a working test deployment through an automated deployment process in six weeks.

Rebuilding the platform on GCP

The product had been built around AWS, but the client needed its environment hosted in GCP. I mapped the AWS architecture to GCP and built a regional GKE environment with private nodes, networking, load balancing, Cloud Armor, Traefik, and Linkerd.

Application infrastructure

The environment also included RabbitMQ, MongoDB Atlas, KEDA, GCS storage and FUSE volumes, Secret Manager, Identity Platform, and the IAM and network controls needed by the application. I created a separate GCP Helm chart for the application services.

Automated client deployment

I wrote the Terraform modules and a client-deployment pipeline that created per-client resources, initialized Atlas and RabbitMQ, created Kubernetes secrets, generated Helm values, and generated the ArgoCD application. It also supported dry runs and repeatable teardown.

Testing and delivery

I built integration checks for the supporting services and used a test tenant to exercise the deployment path and application configuration. Within six weeks, the automated process produced a working GCP environment and a test deployment.

Impact

The working test deployment showed the client that BlackBoiler could meet its GCP hosting requirement. It also made the product available to other clients that required GCP, with an automated process that could be reused for future environments.

Technologies & skills

  • GCP
  • GKE
  • Kubernetes
  • Terraform
  • Cloud Armor
  • Traefik
  • Linkerd
  • KEDA
  • GCS
  • MongoDB Atlas
  • RabbitMQ
  • Helm
  • ArgoCD
  • Integration testing

BlackBoiler / AI assistant interface

Turning AI answers into interactive analytics

  • AI / ML
  • Product engineering
  • Security

I built the interactive canvas for BlackBoiler’s in-app assistant, allowing an agent response to include live charts and tables instead of limiting every answer to text.

The product gap

The assistant could answer questions about subscriptions, contracts, playbooks, rule performance, and edit acceptance, but it returned everything as prose. That made comparisons and trends harder to understand than the same data shown visually.

Widget intents

I registered widget intents that the Server-Sent Events stream can invoke while an answer is arriving. Each intent renders an existing chart or table component and reuses the product’s analytics calculations instead of creating a second implementation in the assistant.

Conversation continuity

Leaving the assistant route used to destroy the chat and canvas even though the server session remained active. I moved the thread state to a module-scoped store so people can navigate elsewhere and return to the same conversation.

Session isolation and testing

I strengthened user-level session isolation, added a recent-activity panel, and built Playwright coverage for all seven widget intents.

Impact

The assistant could present comparisons and trends in the format best suited to the data while preserving conversation state across navigation. User-scoped sessions and end-to-end tests made the feature safe to use in a multi-tenant product.

Technologies & skills

  • React
  • TypeScript
  • AI agents
  • Server-Sent Events
  • Data visualization
  • Session isolation
  • State management
  • Playwright
  • End-to-end testing

Leidos / DARPA MUSE / shared research infrastructure

Building infrastructure for a 22 TB software corpus

  • Platform
  • R&D

As the evaluation and infrastructure PI for DARPA MUSE, I led the design and implementation of a system that organized roughly 22 TB of source code into a structured corpus. Research teams could search and analyze the corpus, build its projects, and access it from their own tools. I coordinated teams with different research methods and got agreement on the corpus, metadata, shared services, and evaluation framework.

A common basis for the research

MUSE asked whether large collections of existing software could help machines find vulnerabilities, repair programs, and synthesize new code. The research teams used very different methods, so the program needed a shared corpus and independent evaluations to make their results reproducible and comparable.

Research infrastructure at scale

By the end of Phase 2, the corpus held 436,674 projects, 121 million source files, and 41.5 billion lines of code in 17.9 TB. We built crawlers, metadata extractors, an Elasticsearch-backed search site, analytics, shared storage, private cloud services, and automated builds so teams could work with the collection as a research system.

Co-design across teams

I ran biweekly working groups to understand what each team needed and decide what belonged in the shared platform. Their input shaped the repository, metadata, and syntax ontologies, along with search filters, corpus artifacts, infrastructure priorities, and evaluation plans. That process led every participating team to adopt the platform.

Making the research testable

I developed tools that inferred how to build previously unseen C, C++, and Java projects; the final system built about 30% of the corpus without hand-written instructions. We also created 15 benchmark problems and 19 later challenge problems, then ran integrations, independent evaluations, and hackathons against them.

Impact

The infrastructure gave MUSE a 22 TB structured software collection and a common technical foundation for the program. Every participating team agreed to use the platform, while retaining the different tools and research methods their work required.

Technologies & skills

  • Research infrastructure
  • Elasticsearch
  • OpenStack
  • Distributed storage
  • Redis
  • Doxygen
  • Ontologies
  • Metadata pipelines
  • C/C++
  • Java
  • Android
  • Automated builds
  • Technical evaluation
  • Cross-team negotiation
View the DARPA program overview

Leidos / ONR AGNES / secure software generation

Generating network software with security in the design

  • Security
  • R&D

I helped research and implement AGNES, a system that turned protocol descriptions, reusable implementation primitives, and CWE-based security rules into working network software.

The research question

We wanted to know whether a protocol specification could become the source for a complete network program. If it could, a standards change or a newly understood weakness could be handled in the generator and its reusable primitives, then carried into newly generated software.

Knowledge representation

We modeled protocols, network concepts, program elements, and target-language implementations as named XML units in an ontology-backed repository. Separating each unit’s specification from its language-specific implementations allowed the same concepts to support different languages and platforms.

A working network program

The generator produced C source for a working RIPv2 routing daemon, including the Linux route-table interface. That result showed the model and primitive library could generate functional protocol software, well beyond a collection of scaffolding or code snippets.

Building in security rules

I worked on structured detection and mitigation rules for CWE classes involving buffer bounds, unsafe functions, integer overflow, initialization, and error handling. Encoding those checks in the generator moved repeatable security decisions into the design of the system and made them reusable across generated programs.

Impact

AGNES produced a working routing daemon and showed that protocol knowledge and security rules could be reused in generated software. Encoding recurring security decisions in the generator made them available across generated applications.

Technologies & skills

  • Program generation
  • Software security
  • C
  • Racket
  • XML
  • RDF
  • Ontologies
  • Network protocols
  • RIPv2
  • Linux networking
  • CWE modeling
  • Secure-by-design engineering
View the ACM publication

Georgia Tech / IEEE INFOCOM 1997 / published research

Choosing the right server at the application layer

  • Distributed systems
  • Platform
  • R&D

I co-authored work on application-layer anycasting, which let clients choose among replicated servers using performance and policy criteria without requiring changes to Internet routers.

The problem

Web services were beginning to replicate content across multiple servers, but each request still needed a useful way to choose among them. Existing approaches offered limited control over what “best” meant when a service cared about response time, geography, cost, or policy.

The architecture

The design introduced anycast domain names and resolvers that mapped a service name to one or more server addresses. Metric-based and policy-based filters made server choice an explicit, configurable part of the service.

Measurement

The work included a prototype and experiments with replicated web servers. Resolver-side metric data supported performance-aware selection, and the evaluation compared the resulting client response times with other server-selection approaches.

A deployable form of anycasting

Putting the resolver at the application layer let services deploy the idea on the Internet as it existed, and filters allowed each service to define its own selection policy. The work made server selection a configurable service with both network measurements and application context.

Impact

Services could choose among replicated servers using their own performance and policy criteria. Moving the decision to the application layer avoided changes to Internet routers.

Technologies & skills

  • Distributed systems
  • Server replication
  • Application-layer protocols
  • DNS
  • Performance measurement
  • Policy-based routing
  • Web services
  • Experimental evaluation
Read the paper

Cigital / IEEE SCAM 2001 / published research

Detecting buffer overflows with constraint solving

  • Security
  • R&D

I researched and built Mjolnir as part of a government-funded project. It was an early static-analysis system that combined program-dependence information with mathematical constraints to find buffer-overflow vulnerabilities in C.

The problem

A dangerous library call did not reveal whether a buffer could actually overflow. The analysis had to follow control flow, data flow, buffer sizes, and string operations along the paths leading to that call.

The analysis

Mjolnir represented the program as a system dependence graph, added range constraints that described how statements and C string functions affected buffer sizes, and solved the collected constraints at potential vulnerability sites.

Working across functions

The analyzer traversed functions from the bottom up and built summaries that carried constraints from callees into their callers. This supported analysis across function boundaries while keeping the prototype's analysis tractable.

From research to practice

The work showed how dependence graphs and constraint solving could move security analysis beyond matching suspicious function names. It contributed to patented vulnerability-detection technology and to the commercial source-analysis work that followed at Cigital.

Impact

Mjolnir showed that static analysis could reason about buffer behavior across functions instead of flagging suspicious calls alone. The work contributed to patented technology and later commercial source-analysis tools.

Technologies & skills

  • C
  • Static analysis
  • Constraint solving
  • System dependence graphs
  • Interprocedural analysis
  • Buffer-overflow detection
  • Program analysis
  • Technology transition
View the IEEE publication View the related patent

Cigital / DARPA DISCEX 2003 / published research

Separating security policy from application code

  • Security
  • R&D
  • Product engineering

I led an early effort to use aspect-oriented programming for software security, including a C language extension and a framework for applying security rules consistently across an application.

The design problem

Security requirements often cross module boundaries. Repeating the same check at many call sites makes omissions and inconsistent fixes likely. We wanted security specialists to define a rule once and have the build apply it wherever the program needed it.

The framework

We built an aspect-oriented extension for C and a framework that kept security policy separate from the main program, then applied it throughout the code during the build. The work covered buffer overruns, race conditions, format strings, secure channels, event ordering, and type safety.

Testing the idea with developers

Developers and QA staff used the framework on an application. Their experience exposed problems in understanding, adopting, tracing, and testing the behavior added during the build, which the language prototype alone had not revealed.

The broader contribution

The project explored a way to package specialized security knowledge so application teams could use it consistently. The evaluation also showed that technical capability was only part of adoption: developers needed clear concepts, traceability, evidence, and tools that fit their existing workflow.

Impact

Security specialists could define a rule once and apply it across an application. The developer study found that teams also needed traceability and tools that fit their existing workflow.

Technologies & skills

  • C
  • Aspect-oriented programming
  • Language design
  • Source transformation
  • Security policy
  • Build integration
  • Developer experience
  • Empirical evaluation
View the IEEE publication