b4uship.com is a security scanner. You give it a GitHub repo, it checks for vulnerabilities, broken patterns, and security anti-practices. It tells you what's wrong before you ship.
So I did the obvious thing: I pointed it at itself.
The results were not great.
A security scanner with security issues in its own codebase is peak irony. But it's also a perfect example of why automated scanning matters -- even the person who builds the scanner misses things when coding at speed.
Here's everything I found, unfiltered.
The Setup
b4uship runs 19 static analysis rules against your code and 11 concurrent checks against your live URLs. It was built in 48 hours with Cursor and Claude Code. The backend runs on Modal.com (Python), the frontend is Next.js 14 with Neon Postgres and Stripe.
I ran the full scan against the b4uship repository itself. Same rules, same analysis. No special treatment.
Finding 1: Production Credentials in the Repo
Severity: Critical
The .env.local file contained real production credentials:
- Neon database connection string (with password)
- GitHub OAuth client secret
- The actual NEXTAUTH_SECRET
These weren't in .env.example -- they were in the actual env file that was accessible in the repo. Anyone with access to the codebase had full database write access.
The irony: b4uship's Rule #3 specifically scans for hardcoded secrets and API keys. It checks for patterns like password=, secret=, and api_key= in source files. But .env.local files are excluded from the scan because they're supposed to be gitignored.
Mine wasn't.
Fix: Added .env.local to .gitignore, rotated all credentials, and added .env* patterns to the scan ruleset.
Finding 2: Any User Can View Any Scan Result
Severity: Critical
The scan results endpoint looked like this:
```
GET /api/scans/[id]
```
No authentication check. No ownership verification. If you knew a scan ID, you could view anyone's scan results -- including the security vulnerabilities found in their code.
The scan IDs were sequential-ish (UUIDs, but still). An attacker could enumerate them and dump every scan result in the database. They'd get:
- Repository URLs
- Security findings
- Code snippets with vulnerabilities highlighted
- Whether the user had paid to unlock results
The irony: The scanner finds authorization issues in other people's code. Its own most critical endpoint had no authorization at all.
Fix: Added session check. Users can only view scans they created.
Finding 3: Stripe Error Messages Leaked to Client
Severity: High
The checkout error handler:
```
catch (error) {
return NextResponse.json(
{ error: "Stripe error: " + error.message }
)
}
```
Stripe errors can contain API configuration details, rate limit information, and internal error codes. All of this was being sent directly to the browser.
This wasn't just the checkout endpoint. The same pattern appeared in:
/api/stripe/verify-- payment verification/api/scan-- scan initiation/api/github/fix-pr-- auto-fix PR creation
Every error handler followed the same AI-generated pattern: catch, forward the raw message. The code came from Claude, and Claude optimizes for helpful error messages -- which is the opposite of what you want in production.
Fix: Generic client messages, detailed server logs.
Finding 4: GitHub URL Validation Was a Joke
Severity: High
The scan endpoint accepted a GitHub URL from the user. The validation:
```
if (!github_url.includes("github.com/")) {
return error("Please provide a valid GitHub URL");
}
```
That's it. includes("github.com/"). An attacker could pass:
https://evil.com/path/github.com/fake-- passes validation- URLs with embedded characters
- Non-HTTPS URLs
This is a classic AI code pattern. The validation looks reasonable in isolation. It handles the happy path. But it doesn't handle any adversarial input.
Fix: Proper URL parsing with new URL(), hostname check against github.com, and path pattern validation.
Finding 5: No Rate Limiting Anywhere
Severity: High
None of the public API endpoints had rate limiting:
/api/scan-- triggers a backend scan (costs compute)/api/url-scan-- hits live URLs (costs bandwidth + looks like a DoS)/api/github/repos-- calls GitHub API (has rate limits we'd burn through)
A single user could spam the scan endpoint and exhaust our Modal.com credits in minutes. Or trigger hundreds of URL checks that look like a DDoS attack coming from our IP.
The irony: The scanner checks if scanned websites have rate limiting configured. It doesn't have rate limiting itself.
Fix: This one's still on the TODO list. Planning to add Upstash rate limiting.
Finding 6: AI-Generated Code Committed to User Repos Without Review
Severity: Medium
b4uship has a "fix it for me" feature. It uses Gemini to generate a security fix, then creates a PR on the user's GitHub repo.
The problem: the AI-generated fix is committed directly without any validation. The only check is whether the output is longer than 10 characters. That's it.
The AI could:
- Generate syntactically invalid code
- Introduce new vulnerabilities while fixing old ones
- Output something completely unrelated
A security tool that commits unreviewed AI code to production repos. Let that sink in.
Fix: Added a disclaimer that AI fixes should be reviewed before merging. But honestly, this feature needs a sandbox/preview mode.
The Scorecard
Finding | Severity | b4uship's Own Rules Catch It?
Credentials in repo | Critical | Partially (checks source, not .env)
Missing auth on endpoints | Critical | No
Error message leakage | High | Yes (Rule #7)
Weak input validation | High | No
No rate limiting | High | No
Unreviewed AI commits | Medium | No
Out of 6 findings, b4uship's own scanner would have caught 1.5 of them.
That's a 25% catch rate on its own codebase. Which, coincidentally, matches the Stanford study on AI code vulnerability rates.
What This Teaches
1. Speed creates blind spots.
b4uship was built in 48 hours. In that timeframe, I was focused on "does it work?" not "is it secure?" Every finding in this list is something I would have caught in a normal code review. But I wasn't doing code reviews. I was vibe coding.
2. Knowing about security doesn't mean you practice it.
I literally built a security scanner. I wrote the rules that check for these patterns. And I still shipped code with these issues. Knowledge doesn't automatically become practice, especially under time pressure.
3. Scan your own stuff first.
Before I told anyone else to use b4uship, I should have run it on b4uship. The cobbler's children have no shoes, and the security scanner has no security.
4. Static analysis catches patterns, not logic.
Most of b4uship's rules are regex-based. They catch hardcoded secrets, broad CORS, and known bad patterns. They don't catch missing authorization checks or logic flaws. That's a fundamental limitation of the approach.
What I'm Doing About It
The critical issues (credentials and missing auth) are fixed. The high-severity issues are being addressed. Rate limiting is on the roadmap.
More importantly, I'm updating b4uship's ruleset based on what I found:
- New rule: Check
.env*files, not just source code - New rule: Flag API endpoints without auth middleware
- New rule: Detect raw error forwarding patterns
- Improved: URL validation check in scanned code
The scanner scanned itself, found problems, and now it's better because of it. That's the whole point.
Try It Yourself
Seriously. Run b4uship on your own repo before you ship. If a security scanner can have security issues, your code definitely does too.
And if you want to see how it was built in the first place, read the 48-hour build log.
ContentsTailor is a micro venture studio. We build products, scan them for security issues, and document the embarrassing results. See all projects or apply to build with us.