Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Conversation

pulkitxm
Copy link

@pulkitxm pulkitxm commented Sep 15, 2025

Add a complete example demonstrating how to integrate tRPC, Prisma, and Firecrawl for building type-safe web scraping APIs with database persistence.

Features

  • Type-safe API with tRPC
  • Database persistence with Prisma ORM
  • Web scraping and crawling with Firecrawl
  • Job management with status tracking
  • RESTful endpoints with Express.js

Summary by cubic

Adds a complete example that integrates tRPC, Prisma, and Firecrawl to build type-safe scraping and crawling APIs with database persistence and job tracking. Ships an Express server, Prisma schema, tRPC routers, and a minimal TypeScript client under examples/trpc-prisma-firecrawl.

  • New Features

    • tRPC routers for scraping and crawling with create, get, list, and delete job procedures.
    • Prisma models for ScrapeJob and CrawlJob with status, result data, errors, and timestamps.
    • Firecrawl wrapper with options for markdown/HTML output, main-content extraction, max pages, and external links.
    • Express server with /trpc and /health endpoints; CORS enabled.
    • README with setup, API examples, and client script; env.example and Prisma SQLite schema.
  • Dependencies

    • Adds @mendable/firecrawl-js, @trpc/server, @trpc/client, @prisma/client, prisma, express, zod, cors, tsx, and TypeScript.

Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

15 issues found across 14 files

Prompt for AI agents (all 15 issues)

Understand the root cause of the following 15 issues and fix them.


<file name="examples/trpc-prisma-firecrawl/prisma/schema.prisma">

<violation number="1" location="examples/trpc-prisma-firecrawl/prisma/schema.prisma:16">
Prefer a Prisma enum for status (e.g., JobStatus) to enforce valid states and improve query ergonomics.</violation>

<violation number="2" location="examples/trpc-prisma-firecrawl/prisma/schema.prisma:26">
Use Prisma&#39;s Json type for JSON fields to preserve type-safety instead of storing serialized JSON in String.</violation>

<violation number="3" location="examples/trpc-prisma-firecrawl/prisma/schema.prisma:47">
Use Prisma&#39;s Json type for JSON fields to preserve type-safety instead of storing serialized JSON in String.</violation>
</file>

<file name="examples/trpc-prisma-firecrawl/README.md">

<violation number="1" location="examples/trpc-prisma-firecrawl/README.md:74">
tRPC mutation body should be wrapped under `input`; posting the raw fields will not match tRPC’s expected request shape.</violation>

<violation number="2" location="examples/trpc-prisma-firecrawl/README.md:81">
tRPC GET examples omit the required JSON-encoded `input` parameter, so the shown curl commands will fail schema parsing.</violation>

<violation number="3" location="examples/trpc-prisma-firecrawl/README.md:86">
tRPC GET example for getJob should pass input via JSON-encoded `input` query parameter.</violation>

<violation number="4" location="examples/trpc-prisma-firecrawl/README.md:91">
tRPC mutation requests must wrap input under the `input` key; raw body without `input` will not be parsed by the tRPC express adapter.</violation>

<violation number="5" location="examples/trpc-prisma-firecrawl/README.md:98">
Wrap mutation payload under `input` for tRPC HTTP requests; current curl example will not match expected input shape.</violation>
</file>

<file name="examples/trpc-prisma-firecrawl/src/routers/crawl.ts">

<violation number="1" location="examples/trpc-prisma-firecrawl/src/routers/crawl.ts:5">
Create and reuse a single PrismaClient instance across routers to avoid multiple open clients and reduce resource usage.</violation>

<violation number="2" location="examples/trpc-prisma-firecrawl/src/routers/crawl.ts:57">
Return a fallback error message here to avoid sending undefined and to align with the DB update.</violation>

<violation number="3" location="examples/trpc-prisma-firecrawl/src/routers/crawl.ts:86">
Use TRPCError with NOT_FOUND instead of a generic Error so clients receive a proper tRPC error code.</violation>
</file>

<file name="examples/trpc-prisma-firecrawl/src/client-example.ts">

<violation number="1" location="examples/trpc-prisma-firecrawl/src/client-example.ts:29">
If error exists but is undefined, this will log &quot;undefined&quot; instead of a fallback; use nullish coalescing to ensure a meaningful message.</violation>

<violation number="2" location="examples/trpc-prisma-firecrawl/src/client-example.ts:58">
If error exists but is undefined, this will log &quot;undefined&quot; instead of a fallback; use nullish coalescing to ensure a meaningful message.</violation>

<violation number="3" location="examples/trpc-prisma-firecrawl/src/client-example.ts:72">
Prefer nullish coalescing over || so a valid length of 0 doesn’t trigger the fallback.</violation>

<violation number="4" location="examples/trpc-prisma-firecrawl/src/client-example.ts:76">
Rule violated: **Enforce Use of Logger Instead of Console for Error Logging**

Use logger.error instead of console.error for error logging, and pass structured context (&quot;&lt;message&gt;&quot;, { error }) rather than multiple arguments.</violation>
</file>


Since this is your first cubic review, here's how it works:

  • cubic automatically reviews your code and comments on bugs and improvements
  • Teach cubic by replying to its comments. cubic learns from your replies and gets better over time
  • Ask questions if you need clarification on any suggestion

React with 👍 or 👎 to teach cubic. Mention @cubic-dev-ai to give feedback, ask questions, or re-run the review.

allowExternalLinks Boolean @default(false)

// Results
pages String? // Store array of scraped pages as JSON string
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Prisma's Json type for JSON fields to preserve type-safety instead of storing serialized JSON in String.

Prompt for AI agents
Address the following comment on examples/trpc-prisma-firecrawl/prisma/schema.prisma at line 47:

<comment>Use Prisma&#39;s Json type for JSON fields to preserve type-safety instead of storing serialized JSON in String.</comment>

<file context>
@@ -0,0 +1,54 @@
+  allowExternalLinks Boolean @default(false)
+  
+  // Results
+  pages       String?  // Store array of scraped pages as JSON string
+  totalPages  Int      @default(0)
+  
</file context>
Fix with Cubic

content String?
markdown String?
html String?
metadata String? // Store additional metadata as JSON string
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Prisma's Json type for JSON fields to preserve type-safety instead of storing serialized JSON in String.

Prompt for AI agents
Address the following comment on examples/trpc-prisma-firecrawl/prisma/schema.prisma at line 26:

<comment>Use Prisma&#39;s Json type for JSON fields to preserve type-safety instead of storing serialized JSON in String.</comment>

<file context>
@@ -0,0 +1,54 @@
+  content     String?
+  markdown    String?
+  html        String?
+  metadata    String?  // Store additional metadata as JSON string
+  
+  // Error information
</file context>
Fix with Cubic


#### Crawl a Website
```bash
curl -X POST http://localhost:3000/trpc/crawl.crawlUrl \
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrap mutation payload under input for tRPC HTTP requests; current curl example will not match expected input shape.

Prompt for AI agents
Address the following comment on examples/trpc-prisma-firecrawl/README.md at line 98:

<comment>Wrap mutation payload under `input` for tRPC HTTP requests; current curl example will not match expected input shape.</comment>

<file context>
@@ -0,0 +1,197 @@
+
+#### Crawl a Website
+```bash
+curl -X POST http://localhost:3000/trpc/crawl.crawlUrl \
+  -H &quot;Content-Type: application/json&quot; \
+  -d &#39;{&quot;baseUrl&quot;: &quot;https://example.com&quot;, &quot;maxPages&quot;: 5}&#39;
</file context>
Fix with Cubic


#### Scrape a URL
```bash
curl -X POST http://localhost:3000/trpc/scrape.scrapeUrl \
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tRPC mutation body should be wrapped under input; posting the raw fields will not match tRPC’s expected request shape.

Prompt for AI agents
Address the following comment on examples/trpc-prisma-firecrawl/README.md at line 74:

<comment>tRPC mutation body should be wrapped under `input`; posting the raw fields will not match tRPC’s expected request shape.</comment>

<file context>
@@ -0,0 +1,197 @@
+
+#### Scrape a URL
+```bash
+curl -X POST http://localhost:3000/trpc/scrape.scrapeUrl \
+  -H &quot;Content-Type: application/json&quot; \
+  -d &#39;{&quot;url&quot;: &quot;https://example.com&quot;, &quot;includeMarkdown&quot;: true}&#39;
</file context>
Fix with Cubic

});

if (!job) {
throw new Error('Job not found');
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use TRPCError with NOT_FOUND instead of a generic Error so clients receive a proper tRPC error code.

Prompt for AI agents
Address the following comment on examples/trpc-prisma-firecrawl/src/routers/crawl.ts at line 86:

<comment>Use TRPCError with NOT_FOUND instead of a generic Error so clients receive a proper tRPC error code.</comment>

<file context>
@@ -0,0 +1,123 @@
+      });
+
+      if (!job) {
+        throw new Error(&#39;Job not found&#39;);
+      }
+
</file context>
Fix with Cubic

console.log('- Status:', jobDetails.status);
console.log('- Created:', jobDetails.createdAt);
console.log('- Title:', jobDetails.title);
console.log('- Content length:', jobDetails.content?.length || 0, 'characters');
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prefer nullish coalescing over || so a valid length of 0 doesn’t trigger the fallback.

Prompt for AI agents
Address the following comment on examples/trpc-prisma-firecrawl/src/client-example.ts at line 72:

<comment>Prefer nullish coalescing over || so a valid length of 0 doesn’t trigger the fallback.</comment>

<file context>
@@ -0,0 +1,83 @@
+      console.log(&#39;- Status:&#39;, jobDetails.status);
+      console.log(&#39;- Created:&#39;, jobDetails.createdAt);
+      console.log(&#39;- Title:&#39;, jobDetails.title);
+      console.log(&#39;- Content length:&#39;, jobDetails.content?.length || 0, &#39;characters&#39;);
+    }
+
</file context>
Fix with Cubic

console.log('Total pages:', crawlResult.job.totalPages);
console.log('Status:', crawlResult.job.status);
} else {
console.log('Crawl failed:', 'error' in crawlResult ? crawlResult.error : 'Unknown error');
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If error exists but is undefined, this will log "undefined" instead of a fallback; use nullish coalescing to ensure a meaningful message.

Prompt for AI agents
Address the following comment on examples/trpc-prisma-firecrawl/src/client-example.ts at line 58:

<comment>If error exists but is undefined, this will log &quot;undefined&quot; instead of a fallback; use nullish coalescing to ensure a meaningful message.</comment>

<file context>
@@ -0,0 +1,83 @@
+      console.log(&#39;Total pages:&#39;, crawlResult.job.totalPages);
+      console.log(&#39;Status:&#39;, crawlResult.job.status);
+    } else {
+      console.log(&#39;Crawl failed:&#39;, &#39;error&#39; in crawlResult ? crawlResult.error : &#39;Unknown error&#39;);
+    }
+
</file context>
Fix with Cubic

console.log('Title:', scrapeResult.job.title);
console.log('Status:', scrapeResult.job.status);
} else {
console.log('Scrape failed:', 'error' in scrapeResult ? scrapeResult.error : 'Unknown error');
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If error exists but is undefined, this will log "undefined" instead of a fallback; use nullish coalescing to ensure a meaningful message.

Prompt for AI agents
Address the following comment on examples/trpc-prisma-firecrawl/src/client-example.ts at line 29:

<comment>If error exists but is undefined, this will log &quot;undefined&quot; instead of a fallback; use nullish coalescing to ensure a meaningful message.</comment>

<file context>
@@ -0,0 +1,83 @@
+      console.log(&#39;Title:&#39;, scrapeResult.job.title);
+      console.log(&#39;Status:&#39;, scrapeResult.job.status);
+    } else {
+      console.log(&#39;Scrape failed:&#39;, &#39;error&#39; in scrapeResult ? scrapeResult.error : &#39;Unknown error&#39;);
+    }
+
</file context>
Fix with Cubic

}

} catch (error) {
console.error('❌ Error:', error);
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rule violated: Enforce Use of Logger Instead of Console for Error Logging

Use logger.error instead of console.error for error logging, and pass structured context ("", { error }) rather than multiple arguments.

Prompt for AI agents
Address the following comment on examples/trpc-prisma-firecrawl/src/client-example.ts at line 76:

<comment>Use logger.error instead of console.error for error logging, and pass structured context (&quot;&lt;message&gt;&quot;, { error }) rather than multiple arguments.</comment>

<file context>
@@ -0,0 +1,83 @@
+    }
+
+  } catch (error) {
+    console.error(&#39;❌ Error:&#39;, error);
+  }
+}
</file context>
Fix with Cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Morty Proxy This is a proxified and sanitized view of the page, visit original site.