+ Vik aRutS0tRt^RIt^RIt^RIt^RIHtHtHtH t H t H t H t H t Ht^RIt^RIHt^RIt^RIt^RIt^RIt^RIt^RIt]P.!R4t]^kRt]!4t] !R4t!RR]P84t!R R ]P84t!R R ]P84t!R R]P84t !RR]P8]],4t!!RR]P84t"!RR]P84t#!RR]P84t$!RR]P84t%!RR]P84t&!RR]P84t'!RR]P84t(!RR ]P84t)!R!R"]P84t*!R#R$]P84t+!R%R&]P84t,!R'R(]P84t-!R)R*]P84t.!R+R,]P84t/!R-R.]P84t0!R/R0]%4t1!R1R2]!],]],4t2!R3R4]P84t3!R5R6]P84t4!R7R8]P84t5!R9R:]P84t6!R;R<]P84t7!R=R>]P84t8!R?R@]P84t9!RARB]P84t:!RCRD]P84t;!RERF]P84t<!RGRH]P8]],4t=!RIRJ]P84t>!RKRL]P84t?!RMRN]P84t@!RORP]P84tA!RQRR]P84tB!RSRT]P84tC!RURV]P84tD!RWRX]P84tE!RYRZ]P84tF!R[R\]P84tG!R]R^]P84tH!R_R`]P84tI!RaRb]P84tJ!RcRd]P84tK!ReRf]P84tL!RgRh]P84tM!RiRj]P84tN!RkRL]P84t?!RlRF]P84t<!RmRn4tO!RoRp4tP!RqRr]O4tQ!RsRt]P4tRR#)va" Firecrawl v1 API Client - Legacy Implementation This module provides the legacy v1 implementation of the Firecrawl SDK. It contains the complete `V1FirecrawlApp` class with all v1 API methods and types for backward compatibility. This is used by the unified client to provide version-specific access patterns like app.v1.scrape_url(). Classes: - V1FirecrawlApp: Legacy v1 client for interacting with the Firecrawl API. - AsyncV1FirecrawlApp: Async version of the v1 client. - CrawlWatcher: WebSocket-based crawl monitoring for v1. N) AnyDictOptionalListUnionCallableLiteralTypeVarGeneric)datetime firecrawlc^RIHp\PP \ 4pV!\PP VR44P4p\P!RV\P4pV'd VP^4P4#R# \d\R4R#i;i))Pathz __init__.pyz"^__version__ = ['\"]([^'\"]*)['\"]z&Failed to get version from __init__.pyN)pathlibrospathdirname__file__join read_textresearchMgroupstrip Exceptionprint)r package_path version_file version_matchs R/home/ubuntu/hermes-agent/venv/lib/python3.14/site-packages/firecrawl/v1/client.py get_versionr"s WW__X.l"'',,|]CDNNPlii E|UWUYUYZm $$Q'--/ /   45 sB B. B..CCTc8a]tRt^PtoRtRtRtV3RltRtVt R#)V1AgentOptionszConfiguration for the agent.FIRE-1NcP<V^8dQh/S[R,;R&S[S[,;R&#)r&modelpromptrrstr)format __classdict__s"r! __annotate__V1AgentOptions.__annotate__Ps& 8 ' SM ) __name__ __module__ __qualname____firstlineno____doc__r)r*__annotate_func____static_attributes____classdictcell__r.s@r!r%r%Ps&'E Fr1r%c4a]tRt^UtoRtRtV3RltRtVtR#)V1AgentOptionsExtract2Configuration for the agent in extract operations.r&c4<V^8dQh/S[R,;R&#r(r&r)r)r-r.s"r!r/"V1AgentOptionsExtract.__annotate__U 8 'r1r2N r3r4r5r6r7r)r8r9r:r;s@r!r=r=U<'Er1r=c0a]tRt^YtoRtV3RltRtVtR#)V1ActionsResultz,Result of actions performed during scraping.cR<V^8dQh/S[S[,;R&S[S[,;R&#)r( screenshotspdfs)rr,)r-r.s"r!r/V1ActionsResult.__annotate__Ys$c s)Or1r2Nr3r4r5r6r7r8r9r:r;s@r!rGrGYs6r1rGc`a]tRt^^toRtRtRt]P!RRR7t V3Rlt Rt Vt R#)V1ChangeTrackingDataz& Data for the change tracking format. Njsonaliasc<V^8dQh/S[S[,;R&S[;R&S[;R&S[S[S[S[3,,;R&S[S[,;R&#)r(previousScrapeAt changeStatus visibilitydiff json_field)rr,rr)r-r.s"r!r/!V1ChangeTrackingData.__annotate__^sZsm*    O  4S> ") Br1r2) r3r4r5r6r7rSrVpydanticFieldrWr8r9r:r;s@r!rNrN^s0'+&*D (t6 BJr1rNca]tRt^htoRtRtRtRtRtRt Rt ] P!RRR7t RtRtRtRtRtRtV3RltRtVtR#)V1FirecrawlDocumentz-Document retrieved or processed by Firecrawl.NrOrPc<V^8dQh/S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[S[,,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R &S[S[,;R &S[S[,;R &S[S[,;R &S[S[,;R &#)r(urlmarkdownhtmlrawHtmllinksextractrW screenshotmetadataactionstitle descriptionchangeTracking)rr,rr#rrGrN)r-r.s"r!r/ V1FirecrawlDocument.__annotate__hs #sm" 3-  c]!  DI % a[ @ $sm"o &- C=#%129r1r2)r3r4r5r6r7r^r_r`rarbrcrYrZrWrdrerfrgrhrir8r9r:r;s@r!r\r\hs[7C"HD!G!%EG&nnT@J $J"H)-GE!%K59Nr1r\c8a]tRt^xtoRtRtRtV3RltRtVt R#)V1LocationConfigz$Location configuration for scraping.Ncb<V^8dQh/S[S[,;R&S[S[S[,,;R&#)r(country languages)rr,r)r-r.s"r!r/V1LocationConfig.__annotate__xs* c]!S ")r1r2) r3r4r5r6r7rnror8r9r:r;s@r!rlrlxs.!G%)Ir1rlc<a]tRt^}toRtRtRtRtV3RltRt Vt R#)V1WebhookConfigzConfiguration for webhooks.Nc<V^8dQh/S[;R&S[S[S[S[3,,;R&S[S[S[S[3,,;R&S[S[S[R,,,;R&#)r(r^headersreevents) completedfailedpagestarted)r,rrrr)r-r.s"r!r/V1WebhookConfig.__annotate__}s` Hd38n %,tCH~&-  T'"JKL MT r1r2) r3r4r5r6r7rtrerur8r9r:r;s@r!rrrr}s%(,G)-HPTF r1rrcda]tRt^toRtRt]P!RRR7tRt Rt V3Rlt Rt Vt R#)V1ChangeTrackingOptionsz"Configuration for change tracking.NschemarPc<V^8dQh/S[S[S[R,,,;R&S[S[,;R&S[S[,;R&S[S[,;R&#)r(modes schema_fieldr*tag)zgit-diffrO)rrrrr,)r-r.s"r!r/$V1ChangeTrackingOptions.__annotate__sS D!345 6=3-F SM  # r1r2)r3r4r5r6r7rrYrZrr*rr8r9r:r;s@r!r|r|s.,9=E"*..X"FL FC r1r|cta]tRt^toRtRtRtRtRtRt Rt Rt Rt Rt RtRtRtRtRtRtRtRtV3RltRtVtR#)V1ScrapeOptions#Parameters for scraping operations.N0ucX<V^8dQh/S[S[S[R,,,;R&S[S[S[S[3,,;R&S[S[S[,,;R&S[S[S[,,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R &S[S[,;R &S[S[,;R &S[S[,;R &S[S[R,,;R &S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&#)r(formatsrt includeTags excludeTagsonlyMainContentwaitFortimeoutlocationmobileskipTlsVerificationremoveBase64ImagesblockAdsproxychangeTrackingOptionsmaxAge storeInCacheparsePDF r_r`racontentrbrdscreenshot@fullPagercrOribasicstealthenhancedauto) rrrrr,boolintrlr|)r-r.s"r!r/V1ScrapeOptions.__annotate__s] d7$_`abid38n %,$s)$+  $s)$+  d^* c]!c]"'(/ TN!"$.!-tn# GBC DK$$;<C SM !"4.'#$tn#%r1r2)r3r4r5r6r7rrtrrrrrrrrrrrrrrrr8r9r:r;s@r!rrsk-eiG(,G'+K'+K&*O!G"G+/H!F*.)-#HGKE?C F#'L#H%r1rc8a]tRt^toRtRtRtV3RltRtVt R#) V1WaitActionz'Wait action to perform during scraping.Ncl<V^8dQh/S[R,;R&S[S[,;R&S[S[,;R&#)r(waittype millisecondsselector)rrrr,)r-r.s"r!r/V1WaitAction.__annotate__s6 &/3-&sm" r1r2) r3r4r5r6r7rrr8r9r:r;s@r!rrs1"&L"H r1rc8a]tRt^toRtRtRtV3RltRtVt R#)V1ScreenshotActionz-Screenshot action to perform during scraping.Ncl<V^8dQh/S[R,;R&S[S[,;R&S[S[,;R&#)r(rdrfullPagequality)rrrr)r-r.s"r!r/V1ScreenshotAction.__annotate__s7 , tn#c]! r1r2) r3r4r5r6r7rrr8r9r:r;s@r!rrs7#H!G r1rc0a]tRt^toRtV3RltRtVtR#) V1ClickActionz(Click action to perform during scraping.c@<V^8dQh/S[R,;R&S[;R&#)r(clickrrrr,)r-r.s"r!r/V1ClickAction.__annotate__s! ' Mr1r2NrLr;s@r!rr2r1rc0a]tRt^toRtV3RltRtVtR#) V1WriteActionz(Write action to perform during scraping.c@<V^8dQh/S[R,;R&S[;R&#)r(writertextr)r-r.s"r!r/V1WriteAction.__annotate__s! '  Ir1r2NrLr;s@r!rrrr1rc0a]tRt^toRtV3RltRtVtR#) V1PressActionz(Press action to perform during scraping.c@<V^8dQh/S[R,;R&S[;R&#)r(pressrkeyr)r-r.s"r!r/V1PressAction.__annotate__s! '  Hr1r2NrLr;s@r!rrrr1rc4a]tRt^toRtRtV3RltRtVtR#)V1ScrollActionz)Scroll action to perform during scraping.Ncj<V^8dQh/S[R,;R&S[R,;R&S[S[,;R&#)r(scrollr directionr)updownr+)r-r.s"r!r/V1ScrollAction.__annotate__s8 ( |$$sm" r1r2) r3r4r5r6r7rr8r9r:r;s@r!rrs3#H r1rc0a]tRt^toRtV3RltRtVtR#)V1ScrapeActionz)Scrape action to perform during scraping.c4<V^8dQh/S[R,;R&#)r(scraperrA)r-r.s"r!r/V1ScrapeAction.__annotate__s ( r1r2NrLr;s@r!rrs3r1rc0a]tRt^toRtV3RltRtVtR#)V1ExecuteJavascriptActionz5Execute javascript action to perform during scraping.c@<V^8dQh/S[R,;R&S[;R&#)r(executeJavascriptrscriptr)r-r.s"r!r/&V1ExecuteJavascriptAction.__annotate__s" % && Kr1r2NrLr;s@r!rrs?r1rc<a]tRt^toRtRtRtRtV3RltRt Vt R#) V1PDFActionz&PDF action to perform during scraping.Nc<V^8dQh/S[R,;R&S[S[R,,;R&S[S[,;R&S[S[,;R&#)r(pdfrr- landscapescale) A0A1A2A3A4A5A6LetterLegalTabloidLedger)rrrfloat)r-r.s"r!r/V1PDFAction.__annotate__sN %. Wmn ov~$  E?! r1r2) r3r4r5r6r7r-rrr8r9r:r;s@r!rrs0rvF $I!E r1rc4a]tRt^toRtRtV3RltRtVtR#)V1ExtractAgentr>r&c4<V^8dQh/S[R,;R&#r@rA)r-r.s"r!r/V1ExtractAgent.__annotate__rCr1r2NrDr;s@r!rrrEr1rcda]tRt^toRtRt]P!RRR7tRt Rt V3Rlt Rt Vt R#) V1JsonConfigzConfiguration for extraction.Nr}rPc<V^8dQh/S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&#)r(r*r systemPromptagent)rr,rr)r-r.s"r!r/V1JsonConfig.__annotate__sH SM 3-F3-&  N #* r1r2)r3r4r5r6r7r*rYrZrrrr8r9r:r;s@r!rrs.' F"*..X"FL"&L&*E r1rcDa]tRt^toRtRtRtRtRtRt V3Rlt Rt Vt R#)V1ScrapeParamsrNc <V^8dQh/S[S[,;R&S[S[,;R&S[S[S[S[S[S[S[S[S[ S[ S[ S[ 3 ,,,;R&S[S[ ,;R&S[S[,;R&#)r(rc jsonOptionsrfrwebhook)rrrrrrrrrrrrrr%rr)r-r.s"r!r/V1ScrapeParams.__annotate__s l #*,'.d5/A=R_anp~AOQjlw"wxyzA  N #*  o &- r1r2) r3r4r5r6r7rcrrfrrr8r9r:r;s@r!rrs,-&*G*.K}AG&*E)-G r1rc<a]tRt^toRtRtRtRtV3RltRt Vt R#)V1ScrapeResponsez"Response from scraping operations.TNc^<V^8dQh/S[;R&S[S[,;R&S[S[,;R&#)r(successwarningerror)rrr,)r-r.s"r!r/V1ScrapeResponse.__annotate__s2 c]! C= r1r2 r3r4r5r6r7rrrr8r9r:r;s@r!rrs,G!GE r1rcDa]tRt^toRtRtRtRtRtRt V3Rlt Rt Vt R#)V1BatchScrapeResponsez&Response from batch scrape operations.NTc<V^8dQh/S[S[,;R&S[S[,;R&S[;R&S[S[,;R&S[S[S[,,;R&#)r(idr^rr invalidURLs)rr,rr)r-r.s"r!r/"V1BatchScrapeResponse.__annotate__sY  #  C=  $s)$+ r1r2) r3r4r5r6r7rr^rrrr8r9r:r;s@r!rrs(0BCGE'+K r1rc8a]tRt^toRtRtRtV3RltRtVt R#)V1BatchScrapeStatusResponsez)Response from batch scrape status checks.TNc<V^8dQh/S[;R&S[R ,;R&S[;R&S[;R&S[;R&S[;R&S[S[,;R&S[S[,;R&# r(rrvstatustotal creditsUsed expiresAtnextdata)scrapingrvrw cancelledrrrr rr,rr\)r-r.s"r!r/(V1BatchScrapeStatusResponse.__annotate__y  B CCN  J    3- " ##r1r2 r3r4r5r6r7rrr8r9r:r;s@r!r r s3G Dr1r cta]tRt^toRtRtRtRtRtRt Rt Rt Rt Rt RtRtRtRtRtRtRtRtV3RltRtVtR#) V1CrawlParamsz#Parameters for crawling operations.Nc,<V^8dQh/S[S[S[,,;R&S[S[S[,,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R &S[S[,;R &S[S[S[S[3,,;R &S[S[,;R &S[S[,;R &S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&#)r( includePaths excludePathsmaxDepthmaxDiscoveryDepthlimitallowBackwardLinkscrawlEntireDomainallowExternalLinks ignoreSitemap scrapeOptionsrdeduplicateSimilarURLsignoreQueryParametersregexOnFullURLdelaymaxConcurrencyallowSubdomains)rrr,rrrrrr)r-r.s"r!r/V1CrawlParams.__annotate__s:49%,49%,sm"   }+  C= !- ~,!-D>(O,3eC01 29%TN1$D>0TN) C=!"SM(#$d^*%r1r2)r3r4r5r6r7rrrrrr r!r"r#r$rr%r&r'r(r)r*r8r9r:r;s@r!rrsj-(,L(,L"H'+E)-(,)-$(M/3M59G-1,0%)NE$(N&*O%r1rc@a]tRtRtoRtRtRtRtRtV3Rlt Rt Vt R#)V1CrawlResponseiz"Response from crawling operations.NTcz<V^8dQh/S[S[,;R&S[S[,;R&S[;R&S[S[,;R&#)r(rr^rr)rr,r)r-r.s"r!r/V1CrawlResponse.__annotate__sC  #  C= r1r2) r3r4r5r6r7rr^rrr8r9r:r;s@r!r-r-s#,BCGE r1r-c8a]tRtRtoRtRtRtV3RltRtVt R#)V1CrawlStatusResponseiz"Response from crawl status checks.TNc<V^8dQh/S[;R&S[R ,;R&S[;R&S[;R&S[;R&S[;R&S[S[,;R&S[S[,;R&#r r)r-r.s"r!r/"V1CrawlStatusResponse.__annotate__rr1r2rr;s@r!r1r1s,G Dr1r1c8a]tRtRtoRtRtRtV3RltRtVt R#) V1CrawlErrori"zA crawl error.Ncv<V^8dQh/S[;R&S[S[,;R&S[;R&S[S[,;R&S[;R&#)r(r timestampr^coder)r,rr )r-r.s"r!r/V1CrawlError.__annotate__"sJ G!( H  3-  J r1r2) r3r4r5r6r7r7r8r8r9r:r;s@r!r5r5"s$(ID r1r5c0a]tRtRtoRtV3RltRtVtR#)V1CrawlErrorsResponsei*z2Response from crawl/batch scrape error monitoring.cR<V^8dQh/S[S[,;R&S[S[,;R&#)r(errors robotsBlocked)rr5r,)r-r.s"r!r/"V1CrawlErrorsResponse.__annotate__*s&  9r1r2NrLr;s@r!r;r;*s<r1r;cPa]tRtRtoRtRtRtRtRtRt Rt Rt Rt V3Rlt RtVtR#) V1MapParamsi/z"Parameters for mapping operations.Nrc<V^8dQh/S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&#) r(rr#includeSubdomains sitemapOnlyrruseIndexr)rr,rrrl)r-r.s"r!r/V1MapParams.__annotate__/s SM D>( ~,  $&  C= c]"tn#'(/r1r2)r3r4r5r6r7rr#rCrDrrrErr8r9r:r;s@r!rArA/s8, F$(M(,"&KE"G#H+/Hr1rAc<a]tRtRtoRtRtRtRtV3RltRt Vt R#) V1MapResponsei:z!Response from mapping operations.TNcn<V^8dQh/S[;R&S[S[S[,,;R&S[S[,;R&#)r(rrbr)rrrr,)r-r.s"r!r/V1MapResponse.__annotate__:s7  DI % C= r1r2) r3r4r5r6r7rrbrr8r9r:r;s@r!rHrH:s+G!%EE r1rHcxa]tRtRtoRtRt]P!RRR7tRt Rt Rt Rt Rt RtRtV3RltRtVtR#)V1ExtractParamsi@z0Parameters for extracting information from URLs.Nr}rPc<V^8dQh/S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R &#) r(r*rrr"enableWebSearchrCorigin showSourcesr$)rr,rrr)r-r.s"r!r/V1ExtractParams.__annotate__@s SM 3-F3-&  !-  d^*  ~, SM $&O,3r1r2)r3r4r5r6r7r*rYrZrrr"rNrCrOrPr$r8r9r:r;s@r!rLrL@sI: F"*..X"FL"&L)-&*O(, F"&K/3Mr1rLcTa]tRtRtoRtRtRtRtRtRt Rt Rt Rt Rt V3RltRtVtR#)V1ExtractResponseiLz!Response from extract operations.NTc*<V^8dQh/S[S[,;R&S[S[R ,,;R&S[S[,;R&S[;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[S[S[3,,;R&S[S[,;R &#) r(rr rrrrrsourcesr processingrvrw) rr,rr rr#rrr)r-r.s"r!r/V1ExtractResponse.__annotate__Ls  W@A BI!(    1+  C=c]!d38n %,#%r1r2)r3r4r5r6r7rr rrrrrrUrr8r9r:r;s@r!rSrSLs<+BEIF$(IGDE!G(,G!%Kr1rScPa]tRtRto^tRtRtRtRtRt Rt Rt Rt V3Rlt RtVtR#) V1SearchParamsiXNenusapii`c"<V^8dQh/S[;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R &S[S[,;R &#) r(queryrtbsfilterlangrnrrOrr$)r,rrr)r-r.s"r!r/V1SearchParams.__annotate__Xs J C= # SM  3-  c]! sm" SM!c]"O,3r1r2)r3r4r5r6rr`rarbrnrrOrr$r8r9r:r;s@r!rZrZXs9EC FD!G"H!F"G/3Mr1rZc<a]tRtRtoRtRtRtRtV3RltRt Vt R#)V1SearchResponseidz Response from search operations.TNcz<V^8dQh/S[;R&S[S[,;R&S[S[,;R&S[S[,;R&#r(rrrr)rrr\rr,)r-r.s"r!r/V1SearchResponse.__annotate__dsE  " ##c]!  C= r1r2rr;s@r!rereds*G!GE r1rec8a]tRtRtoRtRtRtV3RltRtVt R#)V1CreditUsageDataikNcz<V^8dQh/S[;R&S[S[,;R&S[S[,;R&S[S[,;R&#)r(remaining_credits plan_creditsbilling_period_startbilling_period_endrrr,)r-r.s"r!r/V1CreditUsageData.__annotate__ksA3-&#3-.! , r1r2) r3r4r5r6rmrnror8r9r:r;s@r!rjrjks"&L*.(, r1rjc,a]tRtRtoV3RltRtVtR#)V1CreditUsageResponseiqc2<V^8dQh/S[;R&S[;R&#r(rr)rrj)r-r.s"r!r/"V1CreditUsageResponse.__annotate__qs M r1r2Nr3r4r5r6r8r9r:r;s@r!rsrsqr1rsc8a]tRtRtoRtRtRtV3RltRtVt R#)V1TokenUsageDataiuNcz<V^8dQh/S[;R&S[S[,;R&S[S[,;R&S[S[,;R&#)r(remaining_tokens plan_tokensrnrorp)r-r.s"r!r/V1TokenUsageData.__annotate__usA#%#3-.! , r1r2) r3r4r5r6r}rnror8r9r:r;s@r!rzrzus!%K*.(, r1rzc,a]tRtRtoV3RltRtVtR#)V1TokenUsageResponsei{c2<V^8dQh/S[;R&S[;R&#ru)rrz)r-r.s"r!r/!V1TokenUsageResponse.__annotate__{s M r1r2Nrwr;s@r!rr{rxr1rc8a]tRtRtoRtRtRtV3RltRtVt R#)V1CreditUsageHistoricalPeriodiNcz<V^8dQh/S[S[,;R&S[S[,;R&S[S[,;R&S[;R&#)r( startDateendDateapiKeyrrr,r)r-r.s"r!r/*V1CreditUsageHistoricalPeriod.__annotate__sA}# c]! SM  r1r2 r3r4r5r6rrrr8r9r:r;s@r!rr#I!G Fr1rc,a]tRtRtoV3RltRtVtR#)V1CreditUsageHistoricalResponseicB<V^8dQh/S[;R&S[S[,;R&#r(rperiods)rrr)r-r.s"r!r/,V1CreditUsageHistoricalResponse.__annotate__s M / 00r1r2Nrwr;s@r!rrrxr1rc8a]tRtRtoRtRtRtV3RltRtVt R#)V1TokenUsageHistoricalPeriodiNcz<V^8dQh/S[S[,;R&S[S[,;R&S[S[,;R&S[;R&#)r(rrr tokensUsedr)r-r.s"r!r/)V1TokenUsageHistoricalPeriod.__annotate__s@}# c]! SM O r1r2rr;s@r!rrrr1rc,a]tRtRtoV3RltRtVtR#)V1TokenUsageHistoricalResponseicB<V^8dQh/S[;R&S[S[,;R&#r)rrr)r-r.s"r!r/+V1TokenUsageHistoricalResponse.__annotate__s M . //r1r2Nrwr;s@r!rrrxr1rc@a]tRtRtoRt^ tRtRtRtV3Rlt Rt Vt R#)V1GenerateLLMsTextParamsz3 Parameters for the LLMs.txt generation operation. FTNc<V^8dQh/S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&#)r(maxUrls showFullTextcache._V1GenerateLLMsTextParams__experimental_stream)rrr)r-r.s"r!r/%V1GenerateLLMsTextParams.__annotate__sIc]  4.(  D> $D>0r1r2) r3r4r5r6r7rrrrr8r9r:r;s@r!rrs( G#(L E,0r1rcHa]tRtRtoRt^tRt^tRtRt Rt V3Rlt Rt Vt R#)V1DeepResearchParamsiz- Parameters for the deep research operation. iNc<V^8dQh/S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&#)r(r timeLimitranalysisPromptr/_V1DeepResearchParams__experimental_streamSteps)rrr,r)r-r.s"r!r/!V1DeepResearchParams.__annotate__sksm  }"  c] SM(3-&!)5r1r2)r3r4r5r6r7rrrrrrr8r9r:r;s@r!rrs2 H"IG$(N"&L15r1rc4a]tRtRtoRtRtV3RltRtVtR#)V1DeepResearchResponseiz, Response from the deep research operation. NcN<V^8dQh/S[;R&S[;R&S[S[,;R&#r(rrrrr,r)r-r.s"r!r/#V1DeepResearchResponse.__annotate__s.M  G  C= r1r2) r3r4r5r6r7rr8r9r:r;s@r!rrs  E r1rc8a]tRtRtoRtRtRtV3RltRtVt R#)V1DeepResearchStatusResponseiz3 Status response from the deep research operation. Nc$<V^8dQh/S[;R&S[S[S[S[3,,;R&S[;R&S[S[,;R&S[;R&S[;R&S[;R&S[S[S[S[3,,;R&S[S[S[S[3,,;R &S[S[,;R &#) r(rrr rr currentDepthr activitiesrU summaries)rrrr,rrr)r-r.s"r!r/)V1DeepResearchStatusResponse.__annotate__sM  4S> ")  K  C=NMT#s(^$$$sCx. !!Cyr1r2) r3r4r5r6r7rrr8r9r:r;s@r!rrs&*DEr1rc8a]tRtRtoRtRtRtV3RltRtVt R#)V1GenerateLLMsTextResponseiz-Response from LLMs.txt generation operations.TNcN<V^8dQh/S[;R&S[;R&S[S[,;R&#rr)r-r.s"r!r/'V1GenerateLLMsTextResponse.__annotate__s-  G C= r1r2) r3r4r5r6r7rrr8r9r:r;s@r!rrs7GE r1rc0a]tRtRtoRtV3RltRtVtR#)$V1GenerateLLMsTextStatusResponseDataiNcB<V^8dQh/S[;R&S[S[,;R&#)r(llmstxt llmsfulltxt)r,r)r-r.s"r!r/1V1GenerateLLMsTextStatusResponseData.__annotate__s L#%r1r2)r3r4r5r6rr8r9r:r;s@r!rrs!%Kr1rc<a]tRtRtoRtRtRtRtV3RltRt Vt R#) V1GenerateLLMsTextStatusResponseiz4Status response from LLMs.txt generation operations.TNc<V^8dQh/S[;R&S[S[,;R&S[R,;R&S[S[,;R&S[;R&#)r(rrr rrrV)rrrrr,)r-r.s"r!r/-V1GenerateLLMsTextStatusResponse.__annotate__sS  7 8? 7 88  C=  N r1r2) r3r4r5r6r7rrrr8r9r:r;s@r!rrs>G;?DE r1rc8a]tRtRtoRtRtRtV3RltRtVt R#)reiz% Response from the search operation. Nc<V^8dQh/S[;R&S[S[S[S[3,,;R&S[S[,;R&S[S[,;R&#rg)rrrr,rr)r-r.s"r!r/rhsMM  tCH~   c]!  C=r1r2) r3r4r5r6r7rrr8r9r:r;s@r!reres "GEr1cta]tRtRtoRtRt]P!RRR7tRt Rt Rt Rt Rt RtV3RltRtVtR#) rLiz' Parameters for the extract operation. Nr}rPFc<V^8dQh/S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[,;R&S[S[S[S[3,,;R&#) r(r*r system_promptallow_external_linksenable_web_searchrN show_sourcesr)rr,rrr)r-r.s"r!r/rQs SM  3-F  C=' #4.0 ~-d^+4.( DcN #*r1r2)r3r4r5r6r7r*rYrZrrrrrNrrr8r9r:r;s@r!rLrLsH!F"*..X"FL#'M+0(-&+O#(L&*Er1c*Va]tRtRtoRtRtRV3RlRlltRRRRR RR RR RR RR RRRRRRRRRRRRRRRRRRRRRRRRRRRRR/V3RlRlltRRR RR!RR"RR#RRRR RR$R/V3R%lR&lltV3R'lR(lt V3R)lR*lt RV3R,lR-llt RV3R.lR/llt R0RR1RR2RR3RRRR4RR5RR6RR7RR$RR8RR9RR:RR;RRRRRR?^R@R/V3RAlRBllt R0RR1RR2RR3RRRR4RR5RR6RR7RR$RR8RR9RR:RR;RRRRRR@R/V3RClRDlltV3RElRFltV3RGlRHltV3RIlRJltR0RR1RR2RR3RRRR4RR5RR6RR7RR$RR8RR9RR:RR;RRRRRR@R/V3RKlRLlltRMRR7RRNRRORRRR RRPRRR/V3RQlRRlltRRRRR RR RR RR RR RRRRRRRRRRRRRRRRRRRRSRR?^R>RRRR@R/V3RTlRUlltRRRRR RR RR RR RR RRRRRRRRRRRRRRRRRRRRSRR>RR@RRR/V3RVlRWlltRRRRR RR RR RR RR RRRRRRRRRRRRRRRRRRRRSRR>RRRR@R/V3RXlRYlltV3RZlR[ltV3R\lR]ltRR^RR_RR`RR6R+RaR+RbR+RSR/V3RclRdllltV3RelRfltRR^RR_RR`RR6R+RaR+RbR+RSR/V3RglRhllltRiRRjRRkRRlR/V3RmlRnlltRiRRjRRkRRlR/V3RolRplltV3RqlRrltRV3RslRtlltRV3RulRvllt RV3RwlRxllt!RV3RylRzllt"V3R{lR|lt#V3R}lR~lt$V3RlRlt%R2RRRRiRRRR`RRRRRRR/V3RlRllt&R2RRRRiRRRR`RRR/V3RlRllt'V3RlRlt(V3RlRlt)Rt*RRlt+Rt,RRlt-Rt.Rt/Rt0RRlt1RRlt2Rt3Vt4R#)V1FirecrawlAppia Legacy v1 Firecrawl client for backward compatibility. This class provides the complete v1 API implementation including: - URL scraping with various formats and options - Website crawling with monitoring capabilities - Batch scraping operations - Search functionality - Data extraction with LLM integration - Deep research capabilities - LLMs.txt generation This is used by the unified client to provide version-specific access through app.v1.method_name() patterns. aSchema contains invalid structure for OpenAI: object type with no 'properties' defined but 'additionalProperties: true' (schema-less dictionary not supported by OpenAI). Please define specific properties for your object. Note: Recursive schemas using '$ref' are supported.NcJ<V^8dQhRS[S[,RS[S[,RR/#)r(api_keyapi_urlreturnN)rr,)r-r.s"r!r/V1FirecrawlApp.__annotate__s0QQ Qx}QX\Qr1c ^T;'g\P!R4VnT;'g\P!RR4VnRVP9d/VPf!\P R4\ R4h\PRVP 24R#) z Initialize the V1FirecrawlApp instance with API key, API URL. Args: api_key (Optional[str]): API key for authenticating with the Firecrawl API. api_url (Optional[str]): Base URL for the Firecrawl API. FIRECRAWL_API_KEYFIRECRAWL_API_URLhttps://api.firecrawl.devzapi.firecrawl.devNz%No API key provided for cloud servicezNo API key providedz)Initialized V1FirecrawlApp with API URL: )rgetenvrrloggerr ValueErrordebug)selfrrs&&&r!__init__V1FirecrawlApp.__init__s@@")),?"@ ]]")),?A\"]  $,, .4<<3G NNB C23 3 @OPr1rrt include_tags exclude_tagsonly_main_contentwait_forrrrrskip_tls_verificationremove_base64_images block_adsr parse_pdfrc json_optionsrfchange_tracking_optionsmax_agestore_in_cachezero_data_retentionc/<V^8dQhRS[RS[S[S[R,,,RS[S[S[S[3,,RS[S[S[,,RS[S[S[,,RS[S[,RS[S[,R S[S[,R S[S[,R S[S[,R S[S[,R S[S[,RS[S[,RS[S[R,,RS[S[,RS[S[,RS[S[,RS[S[S[ S[ S[ S[ S[ S[S[S[S[S[3 ,,,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,/#)r(r^rrcrtrrrrrrrrrrrrrrfrrrrrrr)r,rrrrrrrlrrrrrrrrrrrr|rr)r-r.s"r!r/rsU7U7U7d7,g$hij U7 d38n- U7 #49- U7#49-U7 (~U7smU7c]U7/0U7TNU7$,D>U7#+4.U7 ~U7 G$JKL!U7" ~#U7$l+%U7&#<0'U7(d57I=ZgivyGIWYrt*$@AB)U7*&..E%F+U7,c]-U7.%TN/U70"*$1U72*#.3U7r1c VPVR4VP4pRVRR\ 2/pV'dVVR&V'dVVR&V'dVVR&V'dVVR&VeVVR &V'dVVR &V'dVVR &V 'dV PR R R7VR&V eV VR&V eV VR&V eV VR&V eV VR&V'dVVR&VeVVR&VexVP V4p\ V\4'd#RV9dVP VR,4VR&\ V\4'dTMVPR R R7VR&VexVP V4p\ V\4'd#RV9dVP VR,4VR&\ V\4'dTMVPR R R7VR&V'd?VUu.uF.p\ V\4'dTMVPR R R7NK0 upVR&V'd/\ V\4'dTMVPR R R7VR&VeVVR&VeVVR&VeVVR&VP V4RV9dGVR,'d8RVR,9d*VP VR,R,4VR,R&RV9dGVR,'d8RVR,9d*VP VR,R,4VR,R&\P!VP R2TTVeVR, ^,MR R 7pVP^8XdlVP4pVPR!4'dR"V9d\R'/VR",B#R#V9d\R$VR#, 24h\R$V 24hVP!VR&4R #uupi \d \R%4hi;i)(ag Scrape and extract content from a URL. Args: url (str): Target URL to scrape formats (Optional[List[Literal["markdown", "html", "rawHtml", "content", "links", "screenshot", "screenshot@fullPage", "extract", "json"]]]): Content types to retrieve (markdown/html/etc) headers (Optional[Dict[str, str]]): Custom HTTP headers include_tags (Optional[List[str]]): HTML tags to include exclude_tags (Optional[List[str]]): HTML tags to exclude only_main_content (Optional[bool]): Extract main content only wait_for (Optional[int]): Wait for a specific element to appear timeout (Optional[int]): Request timeout (ms) location (Optional[LocationConfig]): Location configuration mobile (Optional[bool]): Use mobile user agent skip_tls_verification (Optional[bool]): Skip TLS verification remove_base64_images (Optional[bool]): Remove base64 images block_ads (Optional[bool]): Block ads proxy (Optional[Literal["basic", "stealth", "auto"]]): Proxy type (basic/stealth) extract (Optional[JsonConfig]): Content extraction settings json_options (Optional[JsonConfig]): JSON extraction settings actions (Optional[List[Union[WaitAction, ScreenshotAction, ClickAction, WriteAction, PressAction, ScrollAction, ScrapeAction, ExecuteJavascriptAction, PDFAction]]]): Actions to perform change_tracking_options (Optional[ChangeTrackingOptions]): Change tracking settings zero_data_retention (Optional[bool]): Whether to delete data after scrape is done Returns: ScrapeResponse with: * Requested content formats * Page metadata * Extraction results * Success/error status Raises: Exception: If scraping fails scrape_urlr^rO python-sdk@rrtrrNrrrTby_alias exclude_nonerrrrrrrr}rcrrfrrrzeroDataRetention /v1/scrape@@rtrOrrrrFailed to scrape URL. Error: +Failed to parse Firecrawl response as JSON.z scrape URLr2)_validate_kwargs_prepare_headersversiondict_ensure_schema_dict isinstanceupdaterequestspostr status_coderOgetrrr _handle_error)rr^rrtrrrrrrrrrrrrrcrrfrrrrkwargs_headers scrape_paramsactionresponse response_jsons&&$$$$$$$$$$$$$$$$$$$$$, r!rV1FirecrawlApp.scrape_urlsc| fl3((* 3  G9- '.M) $ '.M) $ +7M- ( +7M- (  (/@M+ , '/M) $ '.M) $ (0 tRV (WM* %  &,M( # ,3HM/ 0  +2FM. /  (1M* % %*M' "  (1M* %  ..w7G'4((X-@$($<$rr`rarbrnscrape_optionsc<V^8dQhRS[RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,R S[S[,R S[/ #) r(r_rr`rarbrnrrrr)r,rrrre)r-r.s"r!r/rs]3]3]3C= ]3 # ]3 SM ]33-]3c]]3sm]3c]]3%_5]3*]3r1c 2VPV R4/p VeW+R&VeW;R&VeWKR&VeW[R&VeWkR&VeW{R&VeWR &V eV PR R R 7V R &V PV 4V PR 4p \ RRV/V Bp V PR R R 7pR\ 2VR&V 'dWR &\ P!VP R2RRVP 2/VR7pVP^8XdeVP4pVPR4'dRV9d \R/VB#RV9d\RVR, 24h\RV 24hVPVR4R# \d \R4hi;i)a Search for content using Firecrawl. Args: query (str): Search query string limit (Optional[int]): Max results (default: 5) tbs (Optional[str]): Time filter (e.g. "qdr:d") filter (Optional[str]): Custom result filter lang (Optional[str]): Language code (default: "en") country (Optional[str]): Country code (default: "us") location (Optional[str]): Geo-targeting timeout (Optional[int]): Request timeout in milliseconds scrape_options (Optional[ScrapeOptions]): Result scraping configuration **kwargs: Additional keyword arguments for future compatibility Returns: SearchResponse: Response containing: * success (bool): Whether request succeeded * data (List[FirecrawlDocument]): Search results * warning (Optional[str]): Warning message if any * error (Optional[str]): Error message if any Raises: Exception: If search fails or response cannot be parsed rNrr`rarbrnrrTrr$ integrationr_rrO /v1/search AuthorizationBearer rtrOrrrzSearch failed. Error: rr2)rrrrrZrrrrrrrOrerrr)rr_rr`rarbrnrrrr search_params _integration final_params params_dictr r s&&$$$$$$$$, r!rV1FirecrawlApp.searchsN fh/   %*' " ?#&%  &,( #  $(& !  '.) $  (0* %  '.) $  %-;-@-@$]a-@-bM/ * V$$((7 &CEC]C "''D'I "-gY 7 H )5 &==||nJ '$ ~&>?    3 & O (   $$Y//Fm4K+?WaCSgij$$S3   3 & O (   $$Y//I4N:K]KK -#&TUbcjUkTl$mnn#&TUbTc$dee   x)F G O MNN O&C ;C *C C"c&<V^8dQhRS[RS[/#r,)rr)r-r.s"r!r/rGs GGTGFdGr1c VP4pVP R2V'dRMR,pVPW24pVP^8XdeVP 4pVP R4'dRV9d \ R /VB#RV9d\RVR, 24h\RV 24hVPVR 4R # \d \R4hi;i) z Get historical token usage (v1).z/v1/team/token-usage/historicalr/r0rrrz-Failed to get historical token usage. Error: rzget token usage historicalNr2) rrr rrOrrrrrr1s&& r!get_token_usage_historical)V1FirecrawlApp.get_token_usage_historicalGs((*=>V`BRfhi$$S3   3 & O (   $$Y//I4N9JMJJ -#&STabiTjSk$lmm#&STaSb$cdd   x)E F O MNN Or4 include_paths exclude_paths max_depthmax_discovery_depthallow_backward_linkscrawl_entire_domainrignore_sitemaprdeduplicate_similar_urlsignore_query_parametersregex_on_full_urlr(allow_subdomainsmax_concurrency poll_intervalidempotency_keyc,<V^8dQhRS[RS[S[S[,,RS[S[S[,,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,R S[S[,R S[S[,R S[S[,R S[S[S[S[3,,R S[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[/#)r(r^r9r:r;r<rr=r>rr?rrr@rArBr(rCrDrrErFr r,rrrrrrrrr1)r-r.s"r!r/r[se@<@< @< S * @<  S * @< C= @<&c]@<}@<'tn@<&d^@<'tn@<!@<!1@<%_ 456@<#+4.@< "*$!@<"$D>#@<$}%@<&#4.'@<("#)@<*&d^+@<, }-@<."#/@<2 3@rr?rrr@rArBr(rCrDrrErFr crawl_paramsrrrrtr rs&&$$$$$$$$$$$$$$$$$$$$, r!rJV1FirecrawlApp.crawl_url[sX~ fk2   $+8L (  $+8L (  '0L $  *0CL, -  $)L !  *0CL, - ! -1EL- .  +1EL- .  %,:L )  %,:,?,?\`,?,aL )  &-L # # /5ML1 2 " .4KL0 1  (->L) *  $)L !  '.>L* +  &-rr?rrr@rArBr(rCrDrrFr r,rrrrrrrrr-)r-r.s"r!r/rsVz<z< z< S * z<  S * z< C= z<&c]z<}z<'tnz<&d^z<'tnz<!z<!1z<%_ 456z<#+4.z< "*$!z<"$D>#z<$}%z<&#4.'z<("#)z<*&d^+z<,"#-z<0 1zrr?rrr@rArBr(rCrDrrFrrOrrrtr s&&$$$$$$$$$$$$$$$$$$$, r!rTV1FirecrawlApp.async_crawl_urls%| f&78   $+8L (  $+8L (  '0L $  *0CL, -  $)L !  *0CL, - ! -1EL- .  +1EL- .  %,:L )  %,:,?,?\`,?,aL )  &-L # # /5ML1 2 " .4KL0 1  (->L) *  $)L !  '.>L* +  &-a P"MOO$`&/2]&_ _ )""LL+LQC)PQ!"s= G=AH H #H= H  HH I (II c&<V^8dQhRS[RS[/#rWr,r;)r-r.s"r!r/rs??S?-B?r1c VP4pVPVP RV R2V4pVP^8Xd\ R/VP 4B#VPVR4R# \ R4h;i)a Returns information about crawl errors. Args: id (str): The ID of the crawl job Returns: V1CrawlErrorsResponse containing: * errors (List[Dict[str, str]]): List of errors with fields: - id (str): Error ID - timestamp (str): When the error occurred - url (str): URL that caused the error - error (str): Error message * robotsBlocked (List[str]): List of URLs blocked by robots.txt Raises: Exception: If error check fails rZ/errorsrzcheck crawl errorsNr2rr rrr;rOrrrrrtr s&& r!check_crawl_errors!V1FirecrawlApp.check_crawl_errorss&'')$$ ~Zt7%KWU   3 & P,?x}}??   x)= > P"MOO A22 A?c<<V^8dQhRS[RS[S[S[3,/#rWr,rr)r-r.s"r!r/rs#==s=tCH~=r1c VP4pVPVP RV 2V4pVP^8XdVP 4#VP VR4R# \ R4h;i)- Cancel an asynchronous crawl job. Args: id (str): The ID of the crawl job to cancel Returns: Dict[str, Any] containing: * success (bool): Whether cancellation was successful * error (str, optional): Error message if cancellation failed Raises: Exception: If cancellation fails rZrzcancel crawl jobN)r_delete_requestrrrOrrros&& r! cancel_crawlV1FirecrawlApp.cancel_crawlsz'')''4<<. 2$(GQ   3 & P}}&   x); < P"MOOs A(( A5c*<V^8dQhRS[RS[S[S[,,RS[S[S[,,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,R S[S[,R S[S[,R S[S[,R S[S[S[S[3,,R S[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RR/#)r(r^r9r:r;r<rr=r>rr?rrr@rArBr(rCrDrrFrV1CrawlWatcher)r,rrrrrrrr)r-r.s"r!r/rsVS9S9S9$DI. S9 $DI. S9  } S9"*#S9C=S9#+4.S9"*$S9#+4.S9%TNS9%_5S9eC$89:S9'/tnS9 &.d^!S9" (~#S9$C=%S9&'tn'S9(&c])S9*"*$+S9,&c]-S90 1S9r1c $VP!V3/RVbRVbRVbRVbRVbRVbRVbRV bR V bR V bR V bR V bR VbRVbRVbRVbRVbRVbRVbVBpVP'd)VP'd\VPV4#\ R4h)am Initiate a crawl job and return a CrawlWatcher to monitor the job via WebSocket. Args: url (str): Target URL to start crawling from include_paths (Optional[List[str]]): Patterns of URLs to include exclude_paths (Optional[List[str]]): Patterns of URLs to exclude max_depth (Optional[int]): Maximum crawl depth max_discovery_depth (Optional[int]): Maximum depth for finding new URLs limit (Optional[int]): Maximum pages to crawl allow_backward_links (Optional[bool]): DEPRECATED: Use crawl_entire_domain instead crawl_entire_domain (Optional[bool]): Follow parent directory links allow_external_links (Optional[bool]): Follow external domain links ignore_sitemap (Optional[bool]): Skip sitemap.xml processing scrape_options (Optional[V1ScrapeOptions]): Page scraping configuration webhook (Optional[Union[str, V1WebhookConfig]]): Notification webhook settings deduplicate_similar_urls (Optional[bool]): Remove similar URLs ignore_query_parameters (Optional[bool]): Ignore URL parameters regex_on_full_url (Optional[bool]): Apply regex to full URLs delay (Optional[int]): Delay in seconds between scrapes allow_subdomains (Optional[bool]): Follow subdomains max_concurrency (Optional[int]): Maximum number of concurrent scrapes zero_data_retention (Optional[bool]): Whether to delete data after 24 hours idempotency_key (Optional[str]): Unique key to prevent duplicate requests **kwargs: Additional parameters to pass to the API Returns: V1CrawlWatcher: An instance to monitor the crawl job via WebSocket Raises: Exception: If crawl job fails to start r9r:r;r<rr=r>rr?rrr@rArBr(rCrDrrFCrawl job failed to start)rTrrr{r)rr^r9r:r;r<rr=r>rr?rrr@rArBr(rCrDrrFrcrawl_responses&&$$$$$$$$$$$$$$$$$$$, r!crawl_url_and_watch"V1FirecrawlApp.crawl_url_and_watchsr--  ' (   !4    "6 !4 "6 * *  &> %< 0 ! ".# $,% &!4' (,+ .  ! ! !n&7&7&7!."3"3T: :78 8r1rinclude_subdomains sitemap_only use_indexc<V^8dQhRS[RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,R S[S[,R S[/ #) r(r^rr?rrrrrrr)r,rrrrlrH)r-r.s"r!r/r7s[0[0[0SM [0 %TN [0 !) [0#4.[0C=[0c][0 ~[0/0[0'[0r1c 6VPV R4/p VeW+R&VeW;R&VeWKR&VeW[R&VeWkR&VeW{R&VeWR &V eV PR R R 7V R &V PV 4V PR 4p \ R/V Bp V PR R R 7pWR&R\ 2VR&V 'dWR &\ P!VP R2RRVP 2/VR7pVP^8XdeVP4pVPR4'dRV9d \R/VB#RV9d\RVR, 24h\RV 24hVPVR4R# \d \R4hi;i)a Map and discover links from a URL. Args: url (str): Target URL to map search (Optional[str]): Filter pattern for URLs ignore_sitemap (Optional[bool]): Skip sitemap.xml processing include_subdomains (Optional[bool]): Include subdomain links sitemap_only (Optional[bool]): Only use sitemap.xml limit (Optional[int]): Maximum URLs to return timeout (Optional[int]): Request timeout in milliseconds **kwargs: Additional parameters to pass to the API Returns: V1MapResponse: Response containing: * success (bool): Whether request succeeded * links (List[str]): Discovered URLs * error (Optional[str]): Error message if any Raises: Exception: If mapping fails or response cannot be parsed map_urlNrr#rCrDrrrETrrrr^rrO/v1/maprrrrrbrzMap failed. Error: rmapr2)rrrrrArrrrrrrOrHrrr)rr^rr?rrrrrrr map_paramsrrrr r s&&$$$$$$$$, r!rV1FirecrawlApp.map_url7sH fi0   #)x  %*8 '  ).@* +  #(4} %  "'w   $+y !  %.z "  %-]]Dt]%TJz " &!!~~m4 #0Z0 "''D'I  E"-gY 7 H )5 &==||nG $$ ~&>?    3 & O (   $$Y//G}4L(9=99 -#&9-:P9Q$RSS#&9-$IJJ   x / O MNN Os &F1F*FFrc.<V^8dQhRS[S[,RS[S[S[R,,,RS[S[S[S[3,,RS[S[S[,,RS[S[S[,,RS[S[,RS[S[,R S[S[,R S[S[,R S[S[,R S[S[,R S[S[,RS[S[,RS[S[R,,RS[S[,RS[S[,RS[S[S[ S[ S[ S[ S[ S[S[S[S[S[3 ,,,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[/#)r(urlsrrcrtrrrrrrrrrrrrrfrrErDrrFr r_r`rarrbrdrrcrOrrr,rrrrrrlrrrrrrrrrrrr%r )r-r.s"r!r/rsKCKC3iKC$w(Q RST KC $sCx.) KC tCy) KCtCy)KC$D>KC3-KC#KC+,KCKC (~KC'tnKCD>KC  FGH!KC",'#KC$|,%KC&$u\3E}VceruCESUnp{&{ |}~'KC(')KC* }+KC,"#-KC.&d^/KC0"#1KC4 %5KCr1c VPVR4/pVeVVR&VeVVR&VeVVR&VeVVR&VeVVR&VeVVR&VeVVR &V eV PR R R 7VR &V eV VR &V eV VR&V eV VR&V eV VR&VeVVR&VexVPV4p\V\4'd#RV9dVPVR,4VR&\V\4'dTMVPR R R 7VR&VexVPV4p\V\4'd#RV9dVPVR,4VR&\V\4'dTMVPR R R 7VR&V'd?VUu.uF.p\V\4'dTMVPR R R 7NK0 upVR&VeVPR R R 7VR&VeVVR&VeVVR&VP V4\ R /VBpVPR R R 7pVVR&R\ 2VR&RV9dGVR,'d8RVR,9d*VPVR,R,4VR,R&RV9dGVR,'d8RVR,9d*VPVR,R,4VR,R&VPV4pVPVP R2VV4pVP^8Xd4VP4PR4pTPTTT4#VPVR4R#uupi \R4h;i)!aH Batch scrape multiple URLs and monitor until completion. Args: urls (List[str]): URLs to scrape formats (Optional[List[Literal]]): Content formats to retrieve headers (Optional[Dict[str, str]]): Custom HTTP headers include_tags (Optional[List[str]]): HTML tags to include exclude_tags (Optional[List[str]]): HTML tags to exclude only_main_content (Optional[bool]): Extract main content only wait_for (Optional[int]): Wait time in milliseconds timeout (Optional[int]): Request timeout in milliseconds location (Optional[LocationConfig]): Location configuration mobile (Optional[bool]): Use mobile user agent skip_tls_verification (Optional[bool]): Skip TLS verification remove_base64_images (Optional[bool]): Remove base64 encoded images block_ads (Optional[bool]): Block advertisements proxy (Optional[Literal]): Proxy type to use extract (Optional[JsonConfig]): Content extraction config json_options (Optional[JsonConfig]): JSON extraction config actions (Optional[List[Union]]): Actions to perform agent (Optional[AgentOptions]): Agent configuration max_concurrency (Optional[int]): Maximum number of concurrent scrapes poll_interval (Optional[int]): Seconds between status checks (default: 2) idempotency_key (Optional[str]): Unique key to prevent duplicate requests **kwargs: Additional parameters to pass to the API Returns: V1BatchScrapeStatusResponse with: * Scraping status and progress * Scraped content for each URL * Success/error information Raises: Exception: If batch scrape fails batch_scrape_urlsNrrtrrrrrTrrrrrrrr}rcrrfrr)rrrrO/v1/batch/scraperrstart batch scrape jobr2)rrrrrrrrrMrrrOrrrNr)rrrrtrrrrrrrrrrrrcrrfrrErDrrFrr r rrr rs&&$$$$$$$$$$$$$$$$$$$$$, r!r V1FirecrawlApp.batch_scrape_urlss@ f&9:   '.M) $  '.M) $  #+7M- (  #+7M- (  (/@M+ ,  '/M) $  '.M) $  (0 tRV (WM* %  &,M( # ,3HM/ 0  +2FM. /  (1M* %  %*M' "  ..w7G'4((X-@$($<$(>8{[dOeCe/3/G/G T]H^_gHh/iK "8 , K 'K ,F,F8WbcpWqKq373K3KKXeLfgoLp3qK &x 0''8%%6F&GV]^   3 & P]]_((.++BG G   x)A BC(U: P"MOOs4M& M++ M8c-<V^8dQhRS[S[,RS[S[S[R,,,RS[S[S[S[3,,RS[S[S[,,RS[S[S[,,RS[S[,RS[S[,R S[S[,R S[S[,R S[S[,R S[S[,R S[S[,RS[S[,RS[S[R,,RS[S[,RS[S[,RS[S[S[ S[ S[ S[ S[ S[S[S[S[S[3 ,,,RS[S[,RS[S[,RS[S[,RS[S[,RS[/#)r(rrrcrtrrrrrrrrrrrrrfrrDrFrrrrrr,rrrrrrlrrrrrrrrrrrr%r)r-r.s"r!r/r!sJCJC3iJC$w(Q RST JC $sCx.) JC tCy) JCtCy)JC$D>JC3-JC#JC+,JCJC (~JC'tnJCD>JC  FGH!JC",'#JC$|,%JC&$u\3E}VceruCESUnp{&{ |}~'JC(')JC*"#+JC,"#-JC.&d^/JC2 3JCr1c VPVR4/pVeVVR&VeVVR&VeVVR&VeVVR&VeVVR&VeVVR&VeVVR &V eV PR R R 7VR &V eV VR &V eV VR&V eV VR&V eV VR&VeVVR&VexVPV4p\V\4'd#RV9dVPVR,4VR&\V\4'dTMVPR R R 7VR&VexVPV4p\V\4'd#RV9dVPVR,4VR&\V\4'dTMVPR R R 7VR&V'd?VUu.uF.p\V\4'dTMVPR R R 7NK0 upVR&VeVPR R R 7VR&VeVVR&VeVVR&VP V4\ R/VBpVPR R R 7pVVR&R\ 2VR&RV9dGVR,'d8RVR,9d*VPVR,R,4VR,R&RV9dGVR,'d8RVR,9d*VPVR,R,4VR,R&VPV4pVPVP R2VV4pVP^8Xd\R/VP4B#VPVR4R#uupi \R4h;i) av Initiate a batch scrape job asynchronously. Args: urls (List[str]): URLs to scrape formats (Optional[List[Literal]]): Content formats to retrieve headers (Optional[Dict[str, str]]): Custom HTTP headers include_tags (Optional[List[str]]): HTML tags to include exclude_tags (Optional[List[str]]): HTML tags to exclude only_main_content (Optional[bool]): Extract main content only wait_for (Optional[int]): Wait time in milliseconds timeout (Optional[int]): Request timeout in milliseconds location (Optional[LocationConfig]): Location configuration mobile (Optional[bool]): Use mobile user agent skip_tls_verification (Optional[bool]): Skip TLS verification remove_base64_images (Optional[bool]): Remove base64 encoded images block_ads (Optional[bool]): Block advertisements proxy (Optional[Literal]): Proxy type to use extract (Optional[JsonConfig]): Content extraction config json_options (Optional[JsonConfig]): JSON extraction config actions (Optional[List[Union]]): Actions to perform agent (Optional[AgentOptions]): Agent configuration max_concurrency (Optional[int]): Maximum number of concurrent scrapes zero_data_retention (Optional[bool]): Whether to delete data after 24 hours idempotency_key (Optional[str]): Unique key to prevent duplicate requests **kwargs: Additional parameters to pass to the API Returns: V1BatchScrapeResponse with: * success - Whether job started successfully * id - Unique identifier for the job * url - Status check URL * error - Error message if start failed Raises: Exception: If job initiation fails async_batch_scrape_urlsNrrtrrrrrTrrrrrrrr}rcrrfrr)rrrrOrrrr2)rrrrrrrrrMrrrrOrr)rrrrtrrrrrrrrrrrrcrrfrrDrFrrr r rrr s&&$$$$$$$$$$$$$$$$$$$$, r!r&V1FirecrawlApp.async_batch_scrape_urls!s@ f&?@   '.M) $  '.M) $  #+7M- (  #+7M- (  (/@M+ ,  '/M) $  '.M) $  (0 tRV (WM* %  &,M( # ,3HM/ 0  +2FM. /  (1M* %  %*M' "  ..w7G'4((X-@$($<$(>8{[dOeCe/3/G/G T]H^_gHh/iK "8 , K 'K ,F,F8WbcpWqKq373K3KKXeLfgoLp3qK &x 0''8%%6F&GV]^   3 & P,?x}}??   x)A BA(U: P"MOOs4M  M Mc-<V^8dQhRS[S[,RS[S[S[R,,,RS[S[S[S[3,,RS[S[S[,,RS[S[S[,,RS[S[,RS[S[,R S[S[,R S[S[,R S[S[,R S[S[,R S[S[,RS[S[,RS[S[R,,RS[S[,RS[S[,RS[S[S[ S[ S[ S[ S[ S[S[S[S[S[3 ,,,RS[S[,RS[S[,RS[S[,RS[S[,RR/#)r(rrrcrtrrrrrrrrrrrrrfrrDrrFrr{rr)rr,rrrrrrlrrrrrrrrrrrr%)r-r.s"r!r/rsJCJC3iJC$w(Q RST JC $sCx.) JC tCy) JCtCy)JC$D>JC3-JC#JC+,JCJC (~JC'tnJCD>JC  FGH!JC",'#JC$|,%JC&$u\3E}VceruCESUnp{&{ |}~'JC(')JC*"#+JC,&d^-JC."#/JC2 3JCr1c NVPVR4/pVeVVR&VeVVR&VeVVR&VeVVR&VeVVR&VeVVR&VeVVR &V eV PR R R 7VR &V eV VR &V eV VR&V eV VR&V eV VR&VeVVR&VexVPV4p\V\4'd#RV9dVPVR,4VR&\V\4'dTMVPR R R 7VR&VexVPV4p\V\4'd#RV9dVPVR,4VR&\V\4'dTMVPR R R 7VR&V'd?VUu.uF.p\V\4'dTMVPR R R 7NK0 upVR&VeVPR R R 7VR&VeVVR&VeVVR&VP V4\ R /VBpVPR R R 7pVVR&R\ 2VR&RV9dGVR,'d8RVR,9d*VPVR,R,4VR,R&RV9dGVR,'d8RVR,9d*VPVR,R,4VR,R&VPV4pVPVP R2VV4pVP^8Xd`\R /VP4BpVP'd)VP'd\VPV4#\!R4hVP#VR4R#uupi \!R4h;i)!a8 Initiate a batch scrape job and return a CrawlWatcher to monitor the job via WebSocket. Args: urls (List[str]): URLs to scrape formats (Optional[List[Literal]]): Content formats to retrieve headers (Optional[Dict[str, str]]): Custom HTTP headers include_tags (Optional[List[str]]): HTML tags to include exclude_tags (Optional[List[str]]): HTML tags to exclude only_main_content (Optional[bool]): Extract main content only wait_for (Optional[int]): Wait time in milliseconds timeout (Optional[int]): Request timeout in milliseconds location (Optional[LocationConfig]): Location configuration mobile (Optional[bool]): Use mobile user agent skip_tls_verification (Optional[bool]): Skip TLS verification remove_base64_images (Optional[bool]): Remove base64 encoded images block_ads (Optional[bool]): Block advertisements proxy (Optional[Literal]): Proxy type to use extract (Optional[JsonConfig]): Content extraction config json_options (Optional[JsonConfig]): JSON extraction config actions (Optional[List[Union]]): Actions to perform agent (Optional[AgentOptions]): Agent configuration max_concurrency (Optional[int]): Maximum number of concurrent scrapes zero_data_retention (Optional[bool]): Whether to delete data after 24 hours idempotency_key (Optional[str]): Unique key to prevent duplicate requests **kwargs: Additional parameters to pass to the API Returns: V1CrawlWatcher: An instance to monitor the batch scrape job via WebSocket Raises: Exception: If batch scrape job fails to start batch_scrape_urls_and_watchNrrtrrrrrTrrrrrrrr}rcrrfrr)rrrrOr Batch scrape job failed to startrrr2)rrrrrrrrrMrrrrOrrr{rr)rrrrtrrrrrrrrrrrrcrrfrrDrrFrr r rrr r~s&&$$$$$$$$$$$$$$$$$$$$, r!r*V1FirecrawlApp.batch_scrape_urls_and_watchsx f&CD   '.M) $  '.M) $  #+7M- (  #+7M- (  (/@M+ ,  '/M) $  '.M) $  (0 tRV (WM* %  &,M( # ,3HM/ 0  +2FM. /  (1M* %  %*M' "  ..w7G'4((X-@$($<$(>8{[dOeCe/3/G/G T]H^_gHh/iK "8 , K 'K ,F,F8WbcpWqKq373K3KKXeLfgoLp3qK &x 0''8%%6F&GV]^   3 & P!6!I!I!)))n.?.?.?).*;*;TBB#$FGG   x)A BI(UB P"MOOs4N UU^_jkr_s^t0u&v v"R!..@PQ 1 #&@g$PQQ""8Y7!6NOOCT#&QSS`&/2]&_ _ *SVS) ) *s['2I'I*I'9I'>I'I!I':A?I' II' I$$I'' J 2JJ c6<V^8dQhRS[RS[S[,/#r(rrr,rSr)r-r.s"r!r/rs!%*%*%*1B31G%*r1c ^RIpVPR\^R7VP4pVP VP RV 2V4pVP ^8Xd\R/VP4B#VPVR4R# \R4h;i \dp\\T4R4hRp?ii;i) a Retrieve the status of an extract job. .. deprecated:: The extract endpoint is in maintenance mode and its use is discouraged. Review https://docs.firecrawl.dev/developer-guides/usage-guides/choosing-the-data-extractor to find a replacement. Args: job_id (str): The ID of the extract job. Returns: ExtractResponse[Any]: The status of the extract job. Raises: ValueError: If there is an error retrieving the status. Nrrrrzget extract statusrr2) rrrrr rrrSrOrrrr,)rrrrtr rgs&& r!get_extract_status!V1FirecrawlApp.get_extract_statuss$   %   '') *((DLL>fX)NPWXH##s*T,?x}}??""8-ABT#&QSS *SVS) ) *s/2B!B:B BB C)B>>Cc<V^8dQhRS[S[S[,,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[S[S[3,,R S[S[,/ #rr)r-r.s"r!r/r;sO*O*49%O*SM O* SM O* $C= O*#+4.O* (~O*#4.O*DcN+O*8I7MO*r1c  0^RIp V PR\^R7VP4p TpV'dVP V4pRVRVRVRVRVR R \ 2/p V'dW+R &V'dWKR &V'dWR &VP VP R2W4p V P^8Xd\R/V P4B#VPV R4R# \R4h;i \dp \\T 4R4hRp ? ii;i)a  Initiate an asynchronous extract job. .. deprecated:: The extract endpoint is in maintenance mode and its use is discouraged. Review https://docs.firecrawl.dev/developer-guides/usage-guides/choosing-the-data-extractor to find a replacement. Args: urls (List[str]): URLs to extract information from prompt (Optional[str]): Custom extraction prompt schema (Optional[Any]): JSON schema/Pydantic model system_prompt (Optional[str]): System context allow_external_links (Optional[bool]): Follow external links enable_web_search (Optional[bool]): Enable web search show_sources (Optional[bool]): Include source URLs agent (Optional[Dict[str, Any]]): Agent configuration idempotency_key (Optional[str]): Unique key to prevent duplicate requests Returns: ExtractResponse[Any] with: * success (bool): Whether request succeeded * data (Optional[Any]): Extracted data matching schema * error (Optional[str]): Error message if any Raises: ValueError: If job initiation fails Nrrrr"rNrPr}rOrr*rrrrz async extractrr2)rrrrrrrMrrrSrOrrrr,)rrr*r}rrrrrrrtrr rgs&&$$$$$$$ r! async_extractV1FirecrawlApp.async_extract;s3N   %   '') --f5F D "6 0 < f  G9-   %+ " +8 ( $) ! *))T\\N+*F ^H##s*T,?x}}??""8_=T#&QSS *SVS) ) *s00C03C C0 C--C00 D;DDmax_urlsshow_full_textrexperimental_streamc ~<V^8dQhRS[RS[S[,RS[S[,RS[S[,RS[S[,RS[/#r(r^rrrrrr,rrrr)r-r.s"r!r/rs`CCCsm C %TN C D> C"*$C<\Cr1c \VVVVR7pVPVVVVVR7pVP'dVP'g\ RRRRR7#VPpVP V4p V P R8XdV #V P R8XdV #V P R 8wd\ RR RRR7#\P!^4Kp) a Generate LLMs.txt for a given URL and poll until completion. Args: url (str): Target URL to generate LLMs.txt from max_urls (Optional[int]): Maximum URLs to process (default: 10) show_full_text (Optional[bool]): Include full text in output (default: False) cache (Optional[bool]): Whether to use cached content if available (default: True) experimental_stream (Optional[bool]): Enable experimental streaming Returns: GenerateLLMsTextStatusResponse with: * Generated LLMs.txt content * Full version if requested * Generation status * Success/error information Raises: Exception: If generation fails rrr__experimental_stream)rrrrFz#Failed to start LLMs.txt generationrwr0rrr rrvrW/LLMs.txt generation job terminated unexpectedly) rasync_generate_llms_textrrrcheck_generate_llms_text_statusr rr) rr^rrrrparamsr rr s &&$$$$ r!generate_llms_text!V1FirecrawlApp.generate_llms_texts8*'"5  00 ) 3 1 x{{{3;  99&AF}} + (* ,.7!K#  JJqMr1c ~<V^8dQhRS[RS[S[,RS[S[,RS[S[,RS[S[,RS[/#rr,rrrr)r-r.s"r!r/rsX8 8 8 sm 8 %TN 8 D> 8 "*$8 bT)JGTH##s*w ( I;HiHH%%, CDD""8-OP0E\emy{||!w#&lmpqrmslt$uvvw  %SV$ $ %s;2CB .C C)CCC C,C''C,cL<V^8dQhRS[S[,RS[S[S[3,/#)r(rFr)rr,r)r-r.s"r!r/r1 s)  %c] 6:38n r1c bV'dRRRRVP 2RV/#RRRRVP 2/#)z Prepare the headers for API requests. Args: idempotency_key (Optional[str]): A unique key to ensure idempotency of requests. Returns: Dict[str, str]: The headers including content type, authorization, and optionally idempotency key. z Content-Typezapplication/jsonrrzx-idempotency-key)r)rrFs&&r!rV1FirecrawlApp._prepare_headers1 sN  274<<.!9#_  . wt||n5  r1c ~<V^8dQhRS[RS[S[S[3,RS[S[S[3,RS[RS[RS[P /#r(r^rrtretriesbackoff_factorr)r,rrrrrResponse)r-r.s"r!r/rI s]sCx.#s(^    "  -5,=,= r1c \V4F~p\P!YTRV9d#VR,eVR,R, ^,MRR7pVPR8Xd'\P !V^V,,4K|Vu# X#)a Make a POST request with retries. Args: url (str): The URL to send the POST request to. data (Dict[str, Any]): The JSON data to include in the POST request. headers (Dict[str, str]): The headers to include in the POST request. retries (int): Number of retries for the request. backoff_factor (float): Backoff factor for retries. Returns: requests.Response: The response from the POST request. Raises: requests.RequestException: If the request fails after the specified retries. rNrr)rangerrrrr)rr^rrtrrattemptr s&&&&&& r!rMV1FirecrawlApp._post_requestI s.W~G}}SqzCrCHLMVHWHcPTU^P_bhPhklPlimoH##s* >Q'\:; & r1c b<V^8dQhRS[RS[S[S[3,RS[RS[RS[P /#r(r^rtrrrr,rrrrr)r-r.s"r!r/rh K#s(^  "  -5,=,= r1c \V4FSp\P!WR7pVPR8Xd'\P !V^V,,4KQVu# X#)a Make a GET request with retries. Args: url (str): The URL to send the GET request to. headers (Dict[str, str]): The headers to include in the GET request. retries (int): Number of retries for the request. backoff_factor (float): Backoff factor for retries. Returns: requests.Response: The response from the GET request. Raises: requests.RequestException: If the request fails after the specified retries. rtr)rrrrrrrr^rtrrrr s&&&&& r!r V1FirecrawlApp._get_requesth sO*W~G||C9H##s* >Q'\:; & r1c b<V^8dQhRS[RS[S[S[3,RS[RS[RS[P /#rr)r-r.s"r!r/r rr1c \V4FSp\P!WR7pVPR8Xd'\P !V^V,,4KQVu# X#)a Make a DELETE request with retries. Args: url (str): The URL to send the DELETE request to. headers (Dict[str, str]): The headers to include in the DELETE request. retries (int): Number of retries for the request. backoff_factor (float): Backoff factor for retries. Returns: requests.Response: The response from the DELETE request. Raises: requests.RequestException: If the request fails after the specified retries. rr)rrdeleterrrrs&&&&& r!rwV1FirecrawlApp._delete_request sO*W~GsQ'\:; & r1cH<V^8dQhRS[RS[S[S[3,RS[RS[/#r(rrtrErr,rrr1)r-r.s"r!r/r sA2J2J2J#s(^2J 2J$9 2Jr1c VP RV 2pVPWB4pVP^8XdVP4pTR,R8XdRT9dTR,pRT9df\ TR,4^8XdMMTPTR,T4pTP4pTP TPR.44KlYvR&\R /TB#\ R4hTR,R 9d&\T^4p\P!T4EK\ RTR, 24hVPVR 4EKF \ R4h;i \ R4h;i) a Monitor the status of a crawl job until completion. Args: id (str): The ID of the crawl job. headers (Dict[str, str]): The headers to include in the status check requests. poll_interval (int): Seconds between status checks. Returns: CrawlStatusResponse: The crawl results if the job is completed successfully. Raises: Exception: If the job fails or an error occurs during status checks. rZrr rvrrz,Crawl job completed but no data was returnedz)Crawl job failed or was stopped. Status: r^r2activepausedpendingqueuedwaitingr) rr rrOrr_r`rr1maxrrr)rrrtrErrercrs&&&& r!rN"V1FirecrawlApp._monitor_job_status sk&j5G"//AO**c1T"1"6"6"8Kx(K7,*62$ 3";v#671< %.2.?.? F@SU\.]O`.=.B.B.D !KK (CD.2F+4C{CC'(VWW *.nn"%mA"6MJJ}-#&OP[\dPeOf$ghh""?4HI1T#&QSS`&/2]&_ _sEE E E$c><V^8dQhRS[PRS[RR/#r(r r rN)rrr,)r-r.s"r!r/r s4#H#H''#H#H!#Hr1c VP4pVPRR4pVPRR4pTP TPY$T4p\PPYqR 7h TPR,pTP4'dRT 2pRTP 2pLRTP 2pR pL \ dR TP 2pR pLi;i;i) a0 Handle errors from API responses. Args: response (requests.Response): The response object from the API request. action (str): Description of the action that was being performed. Raises: Exception: An exception with a message containing the status code and error details from the response. rNo error message provided.details%No additional error details provided.:NiNz#Server returned non-JSON response: zFull response status: z+Server returned empty response with status zNo additional details availablez0Server returned unreadable response with status )r ) rOrrrrr_get_error_messager exceptions HTTPError)rr r r  error_message error_details response_textmessages&&& r!rV1FirecrawlApp._handle_error s B$MMOM)--g7STM)--i9`aM))(*>*>Wde!!++G+GG# B B ( d 3  &&((&I-$YM&O$44V&?#Y&0$Y/0A0BC )Ff%*$'y(9$:!h;. !X-"? 7@S?T UVV!\1 JJqM5'+VWWr1c<V^8dQhRS[RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[S[3,/#) r(r_r;r#rr$rr%rr,rrrrr)r-r.s"r!r/ru sEDEDED } ED ! ED sm ED&c]ED$C=ED*2$EDDHS>EDr1c /pVeW(R&VeW8R&VeWHR&VeWXR&VeWhR&VeWxR&\R/VBpVP4p RV/VPR R R 7Cp R \ 2V R &R V 9dSV R ,p V 'dBRV 9d;\ V R,R4'd"V R,P 4V R ,R&VP VP R2W4p V P^8XdV P4#VPV R4RRRR/# \R4h;i \dp \\T 44hRp ? ii;i)2 Initiates an asynchronous deep research operation. Args: query (str): Research query or topic to investigate max_depth (Optional[int]): Maximum depth of research exploration time_limit (Optional[int]): Time limit in seconds for research max_urls (Optional[int]): Maximum number of URLs to process analysis_prompt (Optional[str]): Custom prompt for analysis system_prompt (Optional[str]): Custom system prompt __experimental_stream_steps (Optional[bool]): Enable experimental streaming Returns: Dict[str, Any]: A response containing: * success (bool): Whether the research initiation was successful * id (str): The unique identifier for the research job * error (str, optional): Error message if initiation failed Raises: Exception: If the research initiation fails. Nrrrrrr,r_TrrrOrr}/v1/deep-researchrzstart deep researchrFrrr2)rrrrhasattrr}rMrrrOrrrr,)rr_r;r#rr$rr%r2rtr json_optsr rgs&&$$$$$$ r!r0"V1FirecrawlApp.async_deep_researchu s>  *3J '  !+5K (  )1I &  &0?, -  $.;N + & 2x5H5O5O5Q -(2 %))T\\N:K*LiaH##s*S#==?*""8-BC5'+BCCS#$QRR %SV$ $ %s0 0D5<D% D5% D22D55 EEEc&<V^8dQhRS[RS[/#rWr,r)r-r.s"r!r/r s (D(DS(D5Q(Dr1c VP4pVPVP RV 2V4pVP^8XdVP 4#VPR8Xd \ R4hVP VR4RRR R /# \ R4h;i \ dp\\T44hRp?ii;i) g Check the status of a deep research operation. Args: id (str): The ID of the deep research operation. Returns: DeepResearchResponse containing: Status: * success - Whether research completed successfully * status - Current state (processing/completed/failed) * error - Error message if failed Results: * id - Unique identifier for the research job * data - Research findings and analysis * sources - List of discovered sources * activities - Research progress log * summaries - Generated research summaries Raises: Exception: If the status check fails. /v1/deep-research/rrzDeep research job not foundzcheck deep research statusNrFrr) rr rrrOrrrr,)rrrtr rgs&& r!r1)V1FirecrawlApp.check_deep_research_status s2'') %((DLL>9KB4)PRYZH##s*S#==?*%%, =>>""8-IJ5'+BCCS#$QRR  %SV$ $ %s/2BB .B BB B>%B99B>c@<V^8dQhRS[S[S[3,RS[RR/#)r(r method_namerNrr,r)r-r.s"r!r/r s/.l.ltCH~.lC.lD.lr1c $V'gR#R0RmR0RmR0RmR0RmR0RmR0RmR0RmR 0Rm/pVPV\44p\VP44V, pV'd"\R V R R P V4 R 24hR#)ah Validate additional keyword arguments before they are passed to the API. This provides early validation before the Pydantic model validation. Args: kwargs (Dict[str, Any]): Additional keyword arguments to validate method_name (str): Name of the method these kwargs are for Raises: ValueError: If kwargs contain invalid or unsupported parameters NrrcrrJrrrrzUnsupported parameter(s) for z: z, zC. Please refer to the API documentation for the correct parameters.>rrrfrcrrrrrrrrrrrrrr> r`rbrrarnrrrr>rrr;rr:r9r?rrBr<r=rrAr@>rrrrrr?r>rr*r}rrrrr>rrrrfrcrrtrrrrrrrrrrr)rsetkeysrr)rrrJ method_paramsallowed_paramsunknown_paramss&&& r!rV1FirecrawlApp._validate_kwargs s  K {  | R "@ &(F *,J% 2'**;>V[[]+n< <[MDIIVdLeKfgjkl l r1c VfV#\V\4'dE\VR4'dVP4#\VR4'dVP 4#\V\ 4'd4VP 4UUu/uFwr#W PV4bK upp#\V\\34'd!VUu.uFq0PV4NK up#V#uuppiuupi)zg Utility to ensure a schema is a dict, not a Pydantic model class. Recursively checks dicts and lists. model_json_schemar}) rrr@rTr}ritemsrlisttuple)rr}kvs&& r!r"V1FirecrawlApp._ensure_schema_dict s >M fd # #v233//11**}}& fd # #?E||~N~tqA//22~N N ftUm , ,9?@A,,Q/@ @ O@s C( C.c V'd\V\\34'gR#Vf \4p^RIpVP !VR\ R7pWd9dR#VPV4\V\4'EdRV9d\VR,\ 4'dVR,PR4p\V4^8dpV^,R8XdbV^,R8XdTVR ,pW8XdVPV4R#W9d+VPW8,W#V4VPV4#VP4F/p VPWW44'gKVPV4R# MK\V\4'd6VF/p VPWW44'gKVPV4R# VPV4R# TPT4i;i) aU Check if an object contains a recursive reference to a specific definition. Args: obj: Object to check target_def_name: Name of the definition to check for recursion defs: Dictionary of definitions visited: Set of visited object keys to detect cycles Returns: True if recursive reference is found, False otherwise FNT) sort_keysdefault$ref/#$defs) rrrVrMrOdumpsr,addsplitr_discard_contains_recursive_refvalues) robjtarget_def_namedefsvisitedrOobj_keyref_pathdef_namevalueitems &&&&& r!rg&V1FirecrawlApp._contains_recursive_ref+ s*S4,77 ?eG**SD#>   G %#t$$S=ZF S%A%A"6{005H8})hqkS.@Xa[T[E[#+B<#6#' OOG $$+#'#?#?P_gn#o OOG $!ZZ\E33EDZZ# OOG $*C&&D33D4YY# OOG $  OOG $ OOG $s1,BG G >(G +G ?3G 7G G G3c V'gR#VP4F wr#VPW2V4'gKR# R#)z Check if $defs contain circular references. Args: defs: Dictionary of definitions to check Returns: True if circular references are found, False otherwise FT)rUrg)rrkro def_values&& r!_check_for_circular_defs'V1FirecrawlApp._check_for_circular_defs] s8#'::< H++IFF$0r1c  V'd$\V\\34'dV^ 8dV#Vf \4p\ V4pWS9dV#VP V4\V\4'EdRV9d\VR,\ 4'dVR,PR4p\V4^8dfV^,R8XdXV^,R8XdJVR,pWr9d;VP\W',4W#V^,4VPV4#VVPV4#/pVP4F)wrV R8XdKVPWW4^,4W&K+ VVPV4#\V\4'd:VU u.uFqPWW4^,4NK up VPV4#VPV4V#uup i TPT4i;i)a$ Resolve $ref references in a JSON schema object. Args: obj: Object to resolve references in defs: Dictionary of definitions visited: Set to track visited objects and prevent infinite recursion depth: Current recursion depth Returns: Object with resolved references r^r_r`rarb) rrrVrMrrdr,rer_ _resolve_refsrfrU) rrirkrldepthobj_idrnroresolvedrrprqs &&&&& r!rxV1FirecrawlApp._resolve_refsp s*S4,7752:J ?eGC  J F $#t$$S=ZF S%A%A"6{005H8})hqkS.@Xa[T[E[#+B<#+#'#5#5d4>6JD[`cd[d#e OOF # OOF #"%))+JCg~ $($6$6uGUVY$WHM#. OOF # C&&WZ[WZt**4w JWZ[ OOF # ' OOF # \ OOF #s2!B6G6)G6.normalize_object sc4((c4((?BCst,T2sCC WF  KK cJ#'!!*$!~~g.!#","2"2"4JC"5$//F%4G.>u.E+.3s+ #5 hNgGY]YcYcYe5fYeQUQRa9I!9L6LYe5fg' v&(2 *56$>56v&(2j( *z*5t<<z,7>>9CJ9O&I9O&+,/G&G',e9ON&I%1?:.&z2":.#:#3#3#56 eT**vU/B&6u&=JOt,,ns&tnsfjDRVAWAW'7'=]a'ans&tJO 7 OOF # wD46g&I'usJ, J1 J7(J7')J<rrrM)rr}rrls&&@@r!_normalize_schema_for_openai+V1FirecrawlApp._normalize_schema_for_openai s3Z55M%> @ ''r1c aaV'd\V\4'gR#\4oVV3RloS!V4'*#)z Validate schema for OpenAI compatibility. Args: schema: Schema to validate Returns: True if schema is valid, False otherwise Tc<\V\4'gR#\V4pVS9dR#SPV4RV9dSP V4R#VP R4R8Xd7RV9d0RV9d)VP R4RJdSP V4R#VP 4Fp\V\4'd,RV9d%S!V4'dSP V4R#KD\V\4'gK\VFGp\V\4'gKRV9gK$S!V4'gK4SP V4R# K SP V4R#)Fr^rrrpatternPropertiesrT)rrrrdrfrrhrV)rirzrprqhas_invalid_structurerls& r!rIV1FirecrawlApp._validate_schema_for_openai..has_invalid_structure s'c4((WF  KK }'8+C'#3../47'eT**vU/B,U33/#4 t,, %%dD11fD6H4T:: ' 7'+ !&& OOF #r1r)rr}rrls&&@@r!_validate_schema_for_openai*V1FirecrawlApp._validate_schema_for_openai s4Z55%# J)000r1c *V'd\V\4'gR#^RIpVP!V4pRV9;'gRV9;'gRV9p\ VP R4;'gVP R44pT;'gT#)z Detect if a schema contains recursive references. Args: schema: Schema to analyze Returns: True if schema has recursive patterns, False otherwise FN"$ref"#/$defs/z#/definitions/ra definitions)rrrOrcrr)rr}rO schema_stringhas_refshas_defss&& r!_detect_recursive_schema'V1FirecrawlApp._detect_recursive_schema) sZ55 6*  % . . - ' . .  -   7+HHvzz-/HI##8#r1c dV'gRRRR/#VPV4'dRRRR/#RRRR/#)z Select appropriate model based on schema complexity. Args: schema: Schema to analyze Returns: Dict with modelName and reason modelNamez gpt-4o-minireason no_schemazgpt-4orecursive_schema_detected simple_schema)r)rr}s&&r!_select_model_for_schema'V1FirecrawlApp._select_model_for_schemaA sF+F F  ( ( 0 085PQ Q]HoFFr1c P\V\4'EdmW!9EdfVPW,4pV'Ed\V\4'dVPR/4p^RIpVP !V4pVPR4;'gRV9;'gRV9pV'dbVP W44pVP !V4p RV 9;'gRV 9p V 'g#Tp\V\4'd RV9dVRM2VP W44p\V\4'd RV9dVRVPV4pVPV4'g\VP4hW1V&V# \dLPi;i \dLai;i)z Process and validate a schema container (like extract or json_options). Args: schema_container: Dict containing schema schema_key: Key where schema is stored (default: "schema") Returns: Processed schema container raNrr) rrrrrOrcrxrrrrOPENAI_SCHEMA_ERROR_MESSAGE) rschema_container schema_keyr}rkrOr has_any_refsresolved_schemaresolved_stringhas_remaining_refss &&& r!_process_schema_with_validation.V1FirecrawlApp._process_schema_with_validationS s & - -*2P--.>.JKFv*VT22zz'2. $ 6 2 JJw'00 -00-/   *.*<*F33F;; !A!ABB+1Z (#%%s01.F F."F1F FF F%$F%rrNN)FN?)Nr)r})5r3r4r5r6r7rrrrr"r)r2r7rJrTrhrprxrrrrrrrrcrrrrrrrMr rwrNrrr9r0r1rrrgrurxrrrrrr9r:r;s@r!rrs $ q QQ$U7mq U7 15 U7 15 U715U715U7'+U7&+U748U7&*U759U748U7)-U7 PT!U7")-#U7$/3%U7&48'U7(EI)U7*JN+U7,&*-U7..2/U70371U7U7n]3$( ]3 "& ]3 %) ]3#']3&*]3'+]3&+]39=]3]3~==,<<,HH(GG(@<.2 @< .2 @< $( @<.2@< $@<04@</3@<04@<*.@<59@<:>@<48@< 37!@<"-1#@<$ $%@<&,0'@<(*.)@<*/3+@<,()-@<.*./@<@z<48z< 37!z<"-1#z<$ $%z<&,0'z<(*.)z<*/3+z<,*.-z<zBS98<S9 7;!S9"15#S9$$(%S9&04'S9(.2)S9*37+S9,.2-S9S9j[0%) [0 .2 [0 26 [0,0[0$([0&+[0)-[048[0[0zKCW[ KC -1 KC -1 KC-1KC-1KC#'KC"'KC04KC"&KC15KC04KC%)KC LP!KC"+/#KC$04%KC&AE'KC(+/)KC*()+KC,*.-KC./3/KC0*.1KCKCZJCW[ JC -1 JC -1 JC-1JC-1JC#'JC"'JC04JC"&JC15JC04JC%)JC LP!JC"+/#JC$04%JC&AE'JC(+/)JC**.+JC,*.-JC./3/JCJCXJCW[ JC -1 JC -1 JC-1JC-1JC#'JC"'JC04JC"&JC15JC04JC%)JC LP!JC"+/#JC$04%JC&AE'JC(+/)JC**.+JC,/3-JC.*./JCJCX<F<F|FF:~P%) ~P %) ~P ,0 ~P49~P16~P,1~P/3~P~P@%*%*NO*%) O* %) O* ,0 O*49O*16O*,1O*/3O*O*bC'+ C .2 C %) C37CCJ8 '+ 8 .2 8 %) 8 378 8 t$}$}L  0>::2J2Jh#H#HJuu4^X(, ^X )- ^X '+ ^X.2^X,0^X;?^XGK^XEI^X^X@ED(, ED )- ED '+ ED.2ED,0ED;?EDEDN(D(DT.l.lf$0d&0dO(b41l$0G$6 6 r1rca]tRtRtoRtV3RlRltV3RlRltV3RlRltV3R lR ltV3R lR lt V3R lRlt Rt Vt R#)r{i a A class to watch and handle crawl job events via WebSocket connection. Attributes: id (str): The ID of the crawl job to watch app (V1FirecrawlApp): The V1FirecrawlApp instance data (List[Dict[str, Any]]): List of crawled documents/data status (str): Current status of the crawl job ws_url (str): WebSocket URL for the crawl job event_handlers (dict): Dictionary of event type to list of handler functions c&<V^8dQhRS[RS[/#r(rapp)r,r)r-r.s"r!r/V1CrawlWatcher.__annotate__ s   3  ^  r1cWnW n.VnRVnVPP RR4 RV 2VnR.R.R./VnR#)rhttpwsrZdonerdocumentN)rrrr rreplacews_urlevent_handlers)rrrs&&&r!rV1CrawlWatcher.__init__ sW*,   ,,VT:;:bTJ B R  r1c<V^8dQhRR/#r(rNr2)r-r.s"r!r/r s * *t *r1c B"\P!VPRRRVPP 23.R7;_uu_4GRjxL pVP V4GRjxL RRR4GRjxL R#L/LL +GRjxL 'giR#;i5i)zE Establishes WebSocket connection and starts listening for messages. Nrr)max_sizeadditional_headers websocketsconnectrrr_listenr websockets& r!rV1CrawlWatcher.connect s%% KK!0GDHH&B* B5B6B>BBB B  B B  Bc<V^8dQhRR/#rr2)r-r.s"r!r/r  , ,$ ,r1c "VRjxL p\P!V4pVPV4GRjxL K;L6LD9R#5i)zq Listens for incoming WebSocket messages and handles them. Args: websocket: The WebSocket connection object NrOloads_handle_messagerrrmsgs&& r!rV1CrawlWatcher._listen E' , ,'**W%C&&s+ + + , +'5AAAA,AAAAAAcV<V^8dQhRS[RS[S[S[S[3,.R3,RR/#)r( event_typehandlerNr)r,rrr)r-r.s"r!r/r s9 < RR?RR@RRARRBRRCRR-^R.R/V3RDlRElltR4RR5RR6RR7RR8RR9RR:RR;RRRR?RR@RRARRBRRCRR-^R.R/V3RFlRGlltV3RHlRIltRV3RJlRKlltRLRR* +4  +4"+4-1cN+4r1c "\P!4;_uu_4GRjxL p\V4FpVPWW4R7;_uu_4GRjxL p V PR8Xd@\ P !V^V,,4GRjxL RRR4GRjxL KxV PR8dVPV RV R24GRjxL V P4GRjxL uuRRR4GRjxL uuuRRR4GRjxL # \R4hELLLLLXLBL3L! +GRjxL 'giEK;i \PdIp Y^, 8XdT h\ P !T^T,,4GRjxL Rp ? EKyRp ? ii;i +GRjxL 'giR#;i5i)a Generic async request method with exponential backoff retry logic. Args: method (str): The HTTP method to use (e.g., "GET" or "POST"). url (str): The URL to send the request to. headers (Dict[str, str]): Headers to include in the request. data (Optional[Dict[str, Any]]): The JSON data to include in the request body (only for POST requests). retries (int): Maximum number of retry attempts (default: 3). backoff_factor (float): Factor to calculate delay between retries (default: 0.5). Delay will be backoff_factor * (2 ** retry_count). Returns: Dict[str, Any]: The parsed JSON response from the server. Raises: aiohttp.ClientError: If the request fails after all retries. Exception: If max retries are exceeded or other errors occur. N)rr^rtrOri,zmake z requestzMax retries exceeded) aiohttp ClientSessionrrequestr asynciorrrO ClientErrorr) rrr^rtrrrsessionrr rgs &&&&&&& r!_async_request"AsyncV1FirecrawlApp._async_request s{6((***g > I&% /   !#??c1")--!w,0O"PPP$     $??c1"&"4"4Xvhh?W"XXX%-]]_4   +**)23 3!+ Q  Y4 +    **IA+-!--!w,(GHHHHI+***s?"GD&GF5ED)E9D7 D+D7  E%D- &E*F5,*D7 D/D7 .D1/D7 2 E?D3 EF5 GD5G F5&G)E+D7 -E/D7 1D7 3E5G7E=E> E E EF5EF2 *6F- F# !F- &F5-F2 2F55G ;F>< G G Gc<V^8dQhRS[RS[S[S[3,RS[S[S[3,RS[RS[RS[S[S[3,/#r)r,rrrr)r-r.s"r!r/r s]^^^"&sCx.^;?S>^^.3^>B38n^r1c F"VPRWW$V4GRjxL #L5i)a Make an async POST request with exponential backoff retry logic. Args: url (str): The URL to send the POST request to. data (Dict[str, Any]): The JSON data to include in the request body. headers (Dict[str, str]): Headers to include in the request. retries (int): Maximum number of retry attempts (default: 3). backoff_factor (float): Factor to calculate delay between retries (default: 0.5). Delay will be backoff_factor * (2 ** retry_count). Returns: Dict[str, Any]: The parsed JSON response from the server. Raises: aiohttp.ClientError: If the request fails after all retries. Exception: If max retries are exceeded or other errors occur. POSTNr)rr^rrtrrs&&&&&&r!_async_post_request'AsyncV1FirecrawlApp._async_post_request s$*((tn]]]] !!c d<V^8dQhRS[RS[S[S[3,RS[RS[RS[S[S[3,/#r)r,rrrr)r-r.s"r!r/r5 sL]]]%)#s(^]].3]>B38n]r1c F"VPRWRW44GRjxL #L5i)ac Make an async GET request with exponential backoff retry logic. Args: url (str): The URL to send the GET request to. headers (Dict[str, str]): Headers to include in the request. retries (int): Maximum number of retry attempts (default: 3). backoff_factor (float): Factor to calculate delay between retries (default: 0.5). Delay will be backoff_factor * (2 ** retry_count). Returns: Dict[str, Any]: The parsed JSON response from the server. Raises: aiohttp.ClientError: If the request fails after all retries. Exception: If max retries are exceeded or other errors occur. GETNr )rr^rtrrs&&&&&r!_async_get_request&AsyncV1FirecrawlApp._async_get_request5 s$(((dG\\\\r c><V^8dQhRS[PRS[RR/#rrClientResponser,)r-r.s"r!r/rK s)++G,B,B+C+TX+r1c R"VP4GRjxL pVPRR4pVPRR4pTP TPY$T4GRjxL p\P!T4hLc \P!RTP 24h;iLD5i)a Handle errors from async API responses with detailed error messages. Args: response (aiohttp.ClientResponse): The response object from the failed request action (str): Description of the action that was being attempted Raises: aiohttp.ClientError: With a detailed error message based on the response status: - 402: Payment Required - 408: Request Timeout - 409: Conflict - 500: Internal Server Error - Other: Unexpected error with status code Nrrrr?Failed to parse Firecrawl error response as JSON. Status code: )rOrrrr _get_async_error_messagerr r  error_datarrrs&&& r!r!AsyncV1FirecrawlApp._handle_errorK s  {'}}.J&NN74PQM&NN96]^M55hoov^kll!!'**/ {%%(ghphwhwgx&yz zls8B'A=A;(A= B' B%!B';A==%B""B'c 8<V^8dQhRS[RS[RS[RS[RS[/#rr)r-r.s"r!r/rf : Z Z# Zs Z[^ Zor Zwz Zr1c ."VPWW44#5ia Generate a standardized error message based on HTTP status code for async operations. Args: status_code (int): The HTTP status code from the response action (str): Description of the action that was being performed error_message (str): The error message from the API response error_details (str): Additional error details from the API response Returns: str: A formatted error message rr!s&&&&&r!r,AsyncV1FirecrawlApp._get_async_error_messagef &&{MYYc <V^8dQhRS[/#rr)r-r.s"r!r/ru s--(=-r1c "VP4pVPVP R2V4GRjxL p\R/VB#L5i)z8Get current credit usage and billing period (v1, async).rNr2)rrrrsrrtresps& r!r"$AsyncV1FirecrawlApp.get_credit_usageu sO''),,||n1 2   %,t,,  3AAAc <V^8dQhRS[/#rr&)r-r.s"r!r/r~ s,,';,r1c "VP4pVPVP R2V4GRjxL p\R/VB#L5i)z7Get current token usage and billing period (v1, async).r(Nr2)rrrrr%s& r!r)#AsyncV1FirecrawlApp.get_token_usage~ sO''),,||n0 1   $+d++  r(cP<V^8dQhRS[RS[S[,RS[S[,RR/#)r(r^rrFrAsyncV1CrawlWatcher)r,rr)r-r.s"r!r/r s=*9*9*9]+*9&c] *97L *9r1c "VPWV4GRjxL pVPR4'dRV9d\VR,V4#\R4hL@5i)a Initiate an async crawl job and return an AsyncV1CrawlWatcher to monitor progress via WebSocket. Args: url (str): Target URL to start crawling from params (Optional[V1CrawlParams]): See V1CrawlParams model for configuration: URL Discovery: * includePaths - Patterns of URLs to include * excludePaths - Patterns of URLs to exclude * maxDepth - Maximum crawl depth * maxDiscoveryDepth - Maximum depth for finding new URLs * limit - Maximum pages to crawl Link Following: * allowBackwardLinks - DEPRECATED: Use crawlEntireDomain instead * crawlEntireDomain - Follow parent directory links * allowExternalLinks - Follow external domain links * ignoreSitemap - Skip sitemap.xml processing Advanced: * scrapeOptions - Page scraping configuration * webhook - Notification webhook settings * deduplicateSimilarURLs - Remove similar URLs * ignoreQueryParameters - Ignore URL parameters * regexOnFullURL - Apply regex to full URLs idempotency_key (Optional[str]): Unique key to prevent duplicate requests Returns: AsyncV1CrawlWatcher: An instance to monitor the crawl job via WebSocket Raises: Exception: If crawl job fails to start Nrrr})rTrr-r)rr^rrFr~s&&&& r!r'AsyncV1FirecrawlApp.crawl_url_and_watch sYL $33CQQ   i ( (T^-C&~d';TB B78 8 RAAA&Ac`<V^8dQhRS[S[,RS[S[,RS[S[,RR/#)r(rrrFrr-)rr,rr)r-r.s"r!r/r sG(@(@s)(@^,(@&c] (@7L (@r1c "VPWV4GRjxL pVPR4'dRV9d\VR,V4#\R4hL@5i)a Initiate an async batch scrape job and return an AsyncV1CrawlWatcher to monitor progress. Args: urls (List[str]): List of URLs to scrape params (Optional[V1ScrapeParams]): See V1ScrapeParams model for configuration: Content Options: * formats - Content formats to retrieve * includeTags - HTML tags to include * excludeTags - HTML tags to exclude * onlyMainContent - Extract main content only Request Options: * headers - Custom HTTP headers * timeout - Request timeout (ms) * mobile - Use mobile user agent * proxy - Proxy type Extraction Options: * extract - Content extraction config * jsonOptions - JSON extraction config * actions - Actions to perform idempotency_key (Optional[str]): Unique key to prevent duplicate requests Returns: AsyncV1CrawlWatcher: An instance to monitor the batch scrape job via WebSocket Raises: Exception: If batch scrape job fails to start Nrrr)rrr-r)rrrrFbatch_responses&&&& r!r/AsyncV1FirecrawlApp.batch_scrape_urls_and_watch sYH $;;D/ZZ   i ( (T^-C&~d';TB B>? ? [r0rrtrrrrrrrrrrrrrrcrrfc/P<V^8dQhRS[RS[S[S[R,,,RS[S[S[S[3,,RS[S[S[,,RS[S[S[,,RS[S[,RS[S[,R S[S[,R S[S[,R S[S[,R S[S[,R S[S[,RS[S[,RS[S[R,,RS[S[,RS[S[,RS[S[,RS[S[S[ S[ S[ S[ S[ S[S[S[S[S[3 ,,,RS[S[,/#)r(r^rrcrtrrrrrrrrrrrrrrfrrr)r,rrrrrrrlrrrrrrrrrrrrr)r-r.s"r!r/r sDMDMDMd7,g$hij DM d38n- DM #49- DM#49-DM (~DMsmDMc]DM/0DMTNDM$,D>DM#+4.DM ~DM G$JKL!DM" ~#DM$l+%DM&#<0'DM(d57I=ZgivyGIWYrt*$@AB)DM**#.+DMr1c "VPVR4VP4pRVRR\ 2/pV'dVVR&V'dVVR&V'dVVR&V'dVVR&VeVVR &V'dVVR &V'dVVR &V 'dV PR R R7VR&V eV VR&V eV VR&V eV VR&V eV VR&V'dVVR&VeVVR&VexVP V4p\ V\4'd#RV9dVP VR,4VR&\ V\4'dTMVPR R R7VR&VexVP V4p\ V\4'd#RV9dVP VR,4VR&\ V\4'dTMVPR R R7VR&V'd?VUu.uF.p\ V\4'dTMVPR R R7NK0 upVR&RV9dGVR,'d8RVR,9d*VP VR,R,4VR,R&RV9dGVR,'d8RVR,9d*VP VR,R,4VR,R&RpVP VP V 2VV4GR jxL pVPR4'dRV9d\R/VR,B#RV9d\RVR, 24hVPR\V44p\RV 24huupiL~5i) a1 Scrape a single URL asynchronously. Args: url (str): Target URL to scrape formats (Optional[List[Literal["markdown", "html", "rawHtml", "content", "links", "screenshot", "screenshot@fullPage", "extract", "json"]]]): Content types to retrieve (markdown/html/etc) headers (Optional[Dict[str, str]]): Custom HTTP headers include_tags (Optional[List[str]]): HTML tags to include exclude_tags (Optional[List[str]]): HTML tags to exclude only_main_content (Optional[bool]): Extract main content only wait_for (Optional[int]): Wait for a specific element to appear timeout (Optional[int]): Request timeout (ms) location (Optional[V1LocationConfig]): Location configuration mobile (Optional[bool]): Use mobile user agent skip_tls_verification (Optional[bool]): Skip TLS verification remove_base64_images (Optional[bool]): Remove base64 images block_ads (Optional[bool]): Block ads proxy (Optional[Literal["basic", "stealth", "auto"]]): Proxy type (basic/stealth) extract (Optional[V1JsonConfig]): Content extraction settings json_options (Optional[V1JsonConfig]): JSON extraction settings actions (Optional[List[Union[V1WaitAction, V1ScreenshotAction, V1ClickAction, V1WriteAction, V1PressAction, V1ScrollAction, V1ScrapeAction, V1ExecuteJavascriptAction, V1PDFAction]]]): Actions to perform **kwargs: Additional parameters to pass to the API Returns: V1ScrapeResponse with: * success - Whether scrape was successful * markdown - Markdown content if requested * html - HTML content if requested * rawHtml - Raw HTML content if requested * links - Extracted links if requested * screenshot - Screenshot if requested * extract - Extracted data if requested * json - JSON data if requested * error - Error message if scrape failed Raises: Exception: If scraping fails rr^rOrrrtrrNrrrTrrrrrrrrr}rcrrfrrrrrr2) rrrrrrr rrrrr,)rr^rrtrrrrrrrrrrrrrcrrfrrr r rbr  error_contents&&$$$$$$$$$$$$$$$$$, r!rAsyncV1FirecrawlApp.scrape_url sz fl3((* 3  G9- '.M) $ '.M) $ +7M- ( +7M- (  (/@M+ , '/M) $ '.M) $ (0 tRV (WM* %  &,M( # ,3HM/ 0  +2FM. /  (1M* % %*M' "  (1M* %  ..w7G'4((X-@$($<$HC3-HC#HC+,HCHC (~HC'tnHCD>HC  FGH!HC",'#HC$|,%HC&$u\3E}VceruCESUnp{&{ |}~'HC(')HC* }+HC,"#-HC0 %1HCr1c "VPVR4/pVeVVR&VeVVR&VeVVR&VeVVR&VeVVR&VeVVR&VeVVR &V eV PR R R 7VR &V eV VR &V eV VR&V eV VR&V eV VR&VeVVR&VexVPV4p\V\4'd#RV9dVPVR,4VR&\V\4'dTMVPR R R 7VR&VexVPV4p\V\4'd#RV9dVPVR,4VR&\V\4'dTMVPR R R 7VR&Ve'VUu.uFpVPR R R 7NK upVR&VeVPR R R 7VR&VP V4\ R/VBpVPR R R 7pVVR&R\ 2VR&RV9dGVR,'d8RVR,9d*VPVR,R,4VR,R&RV9dGVR,'d8RVR,9d*VPVR,R,4VR,R&VPV4pVPVP R2VV4GRjxL pVPR4'd.VPR4pTPTTT4GRjxL #VPVR4R#uupiLa \R4h;iL/5i) a Asynchronously scrape multiple URLs and monitor until completion. Args: urls (List[str]): URLs to scrape formats (Optional[List[Literal]]): Content formats to retrieve headers (Optional[Dict[str, str]]): Custom HTTP headers include_tags (Optional[List[str]]): HTML tags to include exclude_tags (Optional[List[str]]): HTML tags to exclude only_main_content (Optional[bool]): Extract main content only wait_for (Optional[int]): Wait time in milliseconds timeout (Optional[int]): Request timeout in milliseconds location (Optional[LocationConfig]): Location configuration mobile (Optional[bool]): Use mobile user agent skip_tls_verification (Optional[bool]): Skip TLS verification remove_base64_images (Optional[bool]): Remove base64 encoded images block_ads (Optional[bool]): Block advertisements proxy (Optional[Literal]): Proxy type to use extract (Optional[JsonConfig]): Content extraction config json_options (Optional[JsonConfig]): JSON extraction config actions (Optional[List[Union]]): Actions to perform agent (Optional[AgentOptions]): Agent configuration poll_interval (Optional[int]): Seconds between status checks (default: 2) idempotency_key (Optional[str]): Unique key to prevent duplicate requests **kwargs: Additional parameters to pass to the API Returns: V1BatchScrapeStatusResponse with: * Scraping status and progress * Scraped content for each URL * Success/error information Raises: Exception: If batch scrape fails rNrrtrrrrrTrrrrrrrr}rcrrfrrrrOrrrrrr2)rrrrrrrrr rrr_async_monitor_job_statusr)rrrrtrrrrrrrrrrrrcrrfrrErFrr r rrr rs&&$$$$$$$$$$$$$$$$$$$, r!r%AsyncV1FirecrawlApp.batch_scrape_urlscsz f&9:   '.M) $  '.M) $  #+7M- (  #+7M- (  (/@M+ ,  '/M) $  '.M) $  (0 tRV (WM* %  &,M( # ,3HM/ 0  +2FM. /  (1M* %  %*M' "  ..w7G'4((X-@$($<$(>8{[dOeCe/3/G/G T]H^_gHh/iK "8 , K 'K ,F,F8WbcpWqKq373K3KKXeLfgoLp3qK &x 0''811||n, -     << " " P\\$'77G]SS S   x)A BC(m(  P"MOOSsQFMM7A5M-A M:A+M%M&MM M)M*M MMrc-|<V^8dQhRS[S[,RS[S[S[R,,,RS[S[S[S[3,,RS[S[S[,,RS[S[S[,,RS[S[,RS[S[,R S[S[,R S[S[,R S[S[,R S[S[,R S[S[,RS[S[,RS[S[R,,RS[S[,RS[S[,RS[S[S[ S[ S[ S[ S[ S[S[S[S[S[3 ,,,RS[S[,RS[S[,RS[S[,RS[/#)r(rrrcrtrrrrrrrrrrrrrfrrrFrrrr)r-r.s"r!r/rsJIJI3iJI$w(Q RST JI $sCx.) JI tCy) JItCy)JI$D>JI3-JI#JI+,JIJI (~JI'tnJID>JI  FGH!JI",'#JI$|,%JI&$u\3E}VceruCESUnp{&{ |}~'JI(')JI*&d^+JI,"#-JI0 1JIr1c "VPVR4/pVeVVR&VeVVR&VeVVR&VeVVR&VeVVR&VeVVR&VeVVR &V eV PR R R 7VR &V eV VR &V eV VR&V eV VR&V eV VR&VeVVR&VexVPV4p\V\4'd#RV9dVPVR,4VR&\V\4'dTMVPR R R 7VR&VexVPV4p\V\4'd#RV9dVPVR,4VR&\V\4'dTMVPR R R 7VR&V'd?VUu.uF.p\V\4'dTMVPR R R 7NK0 upVR&VeVPR R R 7VR&VeVVR&VP V4\ R/VBpVPR R R 7pVVR&R\ 2VR&RV9dGVR,'d8RVR,9d*VPVR,R,4VR,R&RV9dGVR,'d8RVR,9d*VPVR,R,4VR,R&VPV4pVPVP R2VV4GRjxL pVPR4^8Xd\R/VP4B#VPVR4GRjxL R#uupiLU \R4h;iL5i) a, Initiate a batch scrape job asynchronously. Args: urls (List[str]): URLs to scrape formats (Optional[List[Literal]]): Content formats to retrieve headers (Optional[Dict[str, str]]): Custom HTTP headers include_tags (Optional[List[str]]): HTML tags to include exclude_tags (Optional[List[str]]): HTML tags to exclude only_main_content (Optional[bool]): Extract main content only wait_for (Optional[int]): Wait time in milliseconds timeout (Optional[int]): Request timeout in milliseconds location (Optional[LocationConfig]): Location configuration mobile (Optional[bool]): Use mobile user agent skip_tls_verification (Optional[bool]): Skip TLS verification remove_base64_images (Optional[bool]): Remove base64 encoded images block_ads (Optional[bool]): Block advertisements proxy (Optional[Literal]): Proxy type to use extract (Optional[JsonConfig]): Content extraction config json_options (Optional[JsonConfig]): JSON extraction config actions (Optional[List[Union]]): Actions to perform agent (Optional[AgentOptions]): Agent configuration zero_data_retention (Optional[bool]): Whether to delete data after 24 hours idempotency_key (Optional[str]): Unique key to prevent duplicate requests **kwargs: Additional parameters to pass to the API Returns: V1BatchScrapeResponse with: * success - Whether job started successfully * id - Unique identifier for the job * url - Status check URL * error - Error message if start failed Raises: Exception: If job initiation fails rNrrtrrrrrTrrrrrrrr}rcrrfrrrrrOrrrrr2)rrrrrrrrr rrrrOrr)rrrrtrrrrrrrrrrrrcrrfrrrFrr r rrr s&&$$$$$$$$$$$$$$$$$$$, r!r+AsyncV1FirecrawlApp.async_batch_scrape_urlss| f&?@   '.M) $  '.M) $  #+7M- (  #+7M- (  (/@M+ ,  '/M) $  '.M) $  (0 tRV (WM* %  &,M( # ,3HM/ 0  +2FM. /  (1M* %  %*M' "  ..w7G'4((X-@$($<$(>8{[dOeCe/3/G/G T]H^_gHh/iK "8 , K 'K ,F,F8WbcpWqKq373K3KKXeLfgoLp3qK &x 0''811||n, -     << &# - P,?x}}??$$X/GH H HE(U,  P"MOO HsWFM4M44MA>M4A M4A+M4 M  M4&M">M4M2 M4" M//M4r9r:r;r<rr=r>rr?rrr@rArBr(rCc(<V^8dQhRS[RS[S[S[,,RS[S[S[,,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,R S[S[,R S[S[,R S[S[,R S[S[S[S[3,,R S[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[/#r(r^r9r:r;r<rr=r>rr?rrr@rArBr(rCrErFrrH)r-r.s"r!r/rzs]uBuB uB S * uB  S * uB C= uB&c]uB}uB'tnuB&d^uB'tnuB!uB!1uB%_ 456uB#+4.uB "*$!uB"$D>#uB$}%uB&#4.'uB( })uB*"#+uB. /uBr1c T"VPVR4/pVeVVR&VeVVR&VeVVR&VeVVR&VeVVR&VeVVR&M VeVVR &V eV VR &V eV VR &V eV PR R R 7VR&V eV VR&V eV VR&VeVVR&VeVVR&VeVVR&VeVVR&VPV4\R/VBpVPR R R 7pVVR&R\ 2VR&VP V4pVP VP R2VV4GRjxL pVPR4'd.VPR4pTPTTT4GRjxL #VPVR4GRjxL R#Ld \R4h;iL2L5i)a  Crawl a website starting from a URL. Args: url (str): Target URL to start crawling from include_paths (Optional[List[str]]): Patterns of URLs to include exclude_paths (Optional[List[str]]): Patterns of URLs to exclude max_depth (Optional[int]): Maximum crawl depth max_discovery_depth (Optional[int]): Maximum depth for finding new URLs limit (Optional[int]): Maximum pages to crawl allow_backward_links (Optional[bool]): DEPRECATED: Use crawl_entire_domain instead crawl_entire_domain (Optional[bool]): Follow parent directory links allow_external_links (Optional[bool]): Follow external domain links ignore_sitemap (Optional[bool]): Skip sitemap.xml processing scrape_options (Optional[V1ScrapeOptions]): Page scraping configuration webhook (Optional[Union[str, V1WebhookConfig]]): Notification webhook settings deduplicate_similar_urls (Optional[bool]): Remove similar URLs ignore_query_parameters (Optional[bool]): Ignore URL parameters regex_on_full_url (Optional[bool]): Apply regex to full URLs delay (Optional[int]): Delay in seconds between scrapes allow_subdomains (Optional[bool]): Follow subdomains poll_interval (Optional[int]): Seconds between status checks (default: 2) idempotency_key (Optional[str]): Unique key to prevent duplicate requests **kwargs: Additional parameters to pass to the API Returns: V1CrawlStatusResponse with: * Crawling status and progress * Crawled page contents * Success/error information Raises: Exception: If crawl fails rJNrrrrrr!r r"r#Trr$rr%r&r'r(r*r^rrOrKrrrrLr2) rrrrrrr rrrr;r)rr^r9r:r;r<rr=r>rr?rrr@rArBr(rCrErFrrOrrrtr rs&&$$$$$$$$$$$$$$$$$$, r!rJAsyncV1FirecrawlApp.crawl_urlzs3v fk2   $+8L (  $+8L (  '0L $  *0CL, -  $)L !  *0CL, - ! -1EL- .  +1EL- .  %,:L )  %,:,?,?\`,?,aL )  &-L # # /5ML1 2 " .4KL0 1  (->L) *  $)L !  '.>L* + F#%4|4 "''D'I  E"-gY 7 H''811 \\N) $k7<< << " " P\\$'77G]SS S$$X/@A A A<  P"MOOS AsHD+F(-F.F( FF(1F$2F( F& F( F!!F(&F(c(<V^8dQhRS[RS[S[S[,,RS[S[S[,,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,R S[S[,R S[S[,R S[S[,R S[S[S[S[3,,R S[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[/#rArR)r-r.s"r!r/rs]sBsB sB S * sB  S * sB C= sB&c]sB}sB'tnsB&d^sB'tnsB!sB!1sB%_ 456sB#+4.sB "*$!sB"$D>#sB$}%sB&#4.'sB( })sB*"#+sB. /sBr1c "/pVeVVR&VeVVR&VeVVR&VeVVR&VeVVR&VeVVR&M VeVVR&V eV VR &V eV VR &V eV PR R R 7VR &V eV VR&V eV VR&VeVVR&VeVVR&VeVVR&VeVVR&VPV4\R/VBpVPR R R 7pVVR&R\ 2VR&VP V4pVP VP R2VV4GRjxL pVPR4'd \R/VB#VPVR4GRjxL R#LC \R4h;iL5i)a Start an asynchronous crawl job. Args: url (str): Target URL to start crawling from include_paths (Optional[List[str]]): Patterns of URLs to include exclude_paths (Optional[List[str]]): Patterns of URLs to exclude max_depth (Optional[int]): Maximum crawl depth max_discovery_depth (Optional[int]): Maximum depth for finding new URLs limit (Optional[int]): Maximum pages to crawl allow_backward_links (Optional[bool]): DEPRECATED: Use crawl_entire_domain instead crawl_entire_domain (Optional[bool]): Follow parent directory links allow_external_links (Optional[bool]): Follow external domain links ignore_sitemap (Optional[bool]): Skip sitemap.xml processing scrape_options (Optional[ScrapeOptions]): Page scraping configuration webhook (Optional[Union[str, WebhookConfig]]): Notification webhook settings deduplicate_similar_urls (Optional[bool]): Remove similar URLs ignore_query_parameters (Optional[bool]): Ignore URL parameters regex_on_full_url (Optional[bool]): Apply regex to full URLs idempotency_key (Optional[str]): Unique key to prevent duplicate requests **kwargs: Additional parameters to pass to the API Returns: V1CrawlResponse with: * success - Whether crawl started successfully * id - Unique identifier for the crawl job * url - Status check URL for the crawl * error - Error message if start failed Raises: Exception: If crawl initiation fails Nrrrrrr!r r"r#Trr$rr%r&r'r(r*r^rrOrKrrrLr2) rrrrrr rrr-rr)rr^r9r:r;r<rr=r>rr?rrr@rArBr(rCrErFrrOrrrtr s&&$$$$$$$$$$$$$$$$$$, r!rT#AsyncV1FirecrawlApp.async_crawl_urls p   $+8L (  $+8L (  '0L $  *0CL, -  $)L !  *0CL, - ! -1EL- .  +1EL- .  %,:L )  %,:,?,?\`,?,aL )  &-L # # /5ML1 2 " .4KL0 1  (->L) *  $)L !  '.>L* + F#%4|4 "''D'I  E"-gY 7 H''811 \\N) $     << " " P&222$$X/@A A A  P"MOO As6DE3EE38 E!E3E1E3! E..E3c&<V^8dQhRS[RS[/#rWrX)r-r.s"r!r/rgs @@3@3H@r1c \"VP4pRV 2pVPVP V 2V4GRjxL pVPR4R8XdRV9dVR,pRV9d\ VR,4^8XdMmVPR4pV'g\ P R4M>VPWb4GRjxL pVPVPR.44TpKWTR&\VPR4VPR4VPR4VPR 4VPR 4VPR4R V9dR MR R7pR V9dVPR 4Vn RV9dVPR4Vn V#ELlL5i)a Check the status and results of an asynchronous crawl job. Args: id (str): Unique identifier for the crawl job Returns: V1CrawlStatusResponse containing: Status Information: * status - Current state (scraping/completed/failed/cancelled) * completed - Number of pages crawled * total - Total pages to crawl * creditsUsed - API credits consumed * expiresAt - Data expiration timestamp Results: * data - List of crawled documents * next - URL for next page of results (if paginated) * success - Whether status check succeeded * error - Error message if failed Raises: Exception: If status check fails rZNr rvrrr[r rrrFT)r r rvrrrr) rrrrr_rrr`r1rr rrrtrbrcrrdrfr s && r!rh&AsyncV1FirecrawlApp.check_crawl_statusgs2'')t$ 33||nXJ '   ??8 $ 3$"6* +;v./14*v6H#'HI&*&=&=h&P PIKK fb 9:"+K&*F#(??8,//'*!ook2# 6!ook2($ 3E  k !(__W5HN [ 'OOF3HMI !Qs)9F,F'A!F,*F,F* CF,*F,cH<V^8dQhRS[RS[S[S[3,RS[RS[/#rr)r-r.s"r!r/rs=)_)_#)_S#X)__b)_lA)_r1c "VPVP RV 2V4GRjxL pVPR4R8XdRV9dVR,pRV9d\VR,4^8XdMmVPR4pV'g\P R4M>VPWb4GRjxL pVP VPR.44TpKWTR&\R /VB#\R4hVPR4R 9d,\P!\V^44GRjxL EK7\R VR, 24hEL'LL!5i) a Monitor the status of an asynchronous job until completion. Args: id (str): The ID of the job to monitor headers (Dict[str, str]): Headers to include in status check requests poll_interval (int): Seconds between status checks (default: 2) Returns: V1CrawlStatusResponse: The job results if completed successfully Raises: Exception: If the job fails or an error occurs during status checks rZNr rvrrr[z&Job completed but no data was returnedz#Job failed or was stopped. Status: r2r) rrrr_rrr`r1rrrr )rrrtrErcrrdrfs&&&& r!r;-AsyncV1FirecrawlApp._async_monitor_job_statuss= $ 7 7<<. 2$/!K x(K7[(&v.D K/{623q8!#.??6#:'"NN+LM!*.*A*A(*T$T  IMM&"$=>&/ *.'0?;??#$LMM*.nnmmC q$9:::"EkRZF[E\ ]^^3%U;s;&EEA!E *E5E6A?2   << " "w(': ,8, ,  8'9J8KLM M8 CD D sCD7D5D79=D7r*r}rFrrc<V^8dQhRS[S[S[,,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[S[S[3,,R S[S[,/ #rr)r-r.s"r!r/r!sdRdR49%dRSM dR SM dR $C= dR#+4.dR (~dR#4.dRDcN+dR8I7MdRr1c "^RIp V PR\^R7VP4p V'gV'g \ R4hV'gV'g \ R4hV'dVP V4pRT;'g.RVRVR VR VR R \ 4 2/p V'dW+R &V'dWKR&V'dWR&VPVP R2V V 4GRjxL p V PR4'dV PR4p V 'g \R4hVPVP RV 2V 4GRjxL pVR,R8Xd \R/VB#VR,R9d \RVR, RVR, 24h\P!^4GRjxL K\RV PR4 24hLLL'5i)a Asynchronously extract structured information from URLs. .. deprecated:: The extract endpoint is in maintenance mode and its use is discouraged. Review https://docs.firecrawl.dev/developer-guides/usage-guides/choosing-the-data-extractor to find a replacement. Args: urls (Optional[List[str]]): URLs to extract from prompt (Optional[str]): Custom extraction prompt schema (Optional[Any]): JSON schema/Pydantic model system_prompt (Optional[str]): System context allow_external_links (Optional[bool]): Follow external links enable_web_search (Optional[bool]): Enable web search show_sources (Optional[bool]): Include source URLs agent (Optional[Dict[str, Any]]): Agent configuration Returns: V1ExtractResponse with: * Structured data matching schema * Source information if requested * Success/error status Raises: ValueError: If prompt/schema missing or extraction fails Nrrrrrr"rNrPr}rOrr*rrrrrrrr rvrrrrr2r)rrrrrrr"r rrrrrSrr)rrr*r}rrrrrrrtrr rrcs&&$$$$$$$ r!rcAsyncV1FirecrawlApp.extract!sN   %   '')fBC CF@A A --f5F DJJB "6 0 < f  KM?3   %+ " +8 ( $) !11||nK (     << " "\\$'F KLL$($;$;||nL9% x(K7,;{;; *.EE#l;x3H2IS^_fSgRh$ijjmmA&&&8g9N8OPQ Q1 'skVPWb4GRjxL pVPVP R.44TpKWTR&\VP R4VP R4VP R4VP R 4VP R 4VP R4R 7pR V9d VR ,VR &RV9d VR,VR&R R V9dR/VC#R/VC#ELVL5i)a Check the status of an asynchronous batch scrape job. Args: id (str): The ID of the batch scrape job Returns: V1BatchScrapeStatusResponse containing: Status Information: * status - Current state (scraping/completed/failed/cancelled) * completed - Number of URLs scraped * total - Total URLs to scrape * creditsUsed - API credits consumed * expiresAt - Data expiration timestamp Results: * data - List of scraped documents * next - URL for next page of results (if paginated) * success - Whether status check succeeded * error - Error message if failed Raises: Exception: If status check fails rNr rvrrr[r rr)r r rvrrrrrFT) rrrr_rrrr`r rIs && r!r-AsyncV1FirecrawlApp.check_batch_scrape_statuss2'')&rd+ 33||nXJ '   x K /$"6* +;v./14*v6H#'HI&*&=&=h&P PIKK fb 9:"+K&*F#.??8,//'*!ook2# 6!ook2(   k ! +G 4HW  [ *62HV  ; 6u   <@   G !Qs)9FFAF*FFCFFc&<V^8dQhRS[RS[/#rWrk)r-r.s"r!r/rs  # :O r1c "VP4pVPVP RV R2V4GRjxL #L5i)a Get information about errors from an asynchronous batch scrape job. Args: id (str): The ID of the batch scrape job Returns: V1CrawlErrorsResponse containing: errors (List[Dict[str, str]]): List of errors with fields: * id (str): Error ID * timestamp (str): When the error occurred * url (str): URL that caused the error * error (str): Error message * robotsBlocked (List[str]): List of URLs blocked by robots.txt Raises: Exception: If error check fails rrmNrrrrrrts&& r!r-AsyncV1FirecrawlApp.check_batch_scrape_errorssJ&''),,||n-bT 9     6?=?c&<V^8dQhRS[RS[/#rWrk)r-r.s"r!r/rs  3 3H r1c "VP4pVPVP RV R2V4GRjxL #L5i)a Get information about errors from an asynchronous crawl job. Args: id (str): The ID of the crawl job Returns: V1CrawlErrorsResponse containing: * errors (List[Dict[str, str]]): List of errors with fields: - id (str): Error ID - timestamp (str): When the error occurred - url (str): URL that caused the error - error (str): Error message * robotsBlocked (List[str]): List of URLs blocked by robots.txt Raises: Exception: If error check fails rZrmNrYrZs&& r!rp&AsyncV1FirecrawlApp.check_crawl_errorssI&''),,||nJrd' 2     r\c<<V^8dQhRS[RS[S[S[3,/#rWrt)r-r.s"r!r/rs#--S-T#s(^-r1c "VP4p\P!4;_uu_4GRjxL pVPVP RV 2VR7;_uu_4GRjxL pVP 4GRjxL uuRRR4GRjxL uuRRR4GRjxL #LsLBL,LL +GRjxL 'giM;iRRR4GRjxL R# +GRjxL 'giR#;i5i)rvNrZr)rrrrrrO)rrrtrr s&& r!rx AsyncV1FirecrawlApp.cancel_crawls'')((***g~~j&Ew~WWW[c%]]_,XWW+**W,X+WWWW+*****s2C>B(C>/C!'B* (C!+B2 ?B, B2  C!B. C! C>"B0#C>*C!,B2 .C!0C>2C 8B;9 C C C! C>CC>!C; 'C*( C; 3C; 5 C>c6<V^8dQhRS[RS[S[,/#rr)r-r.s"r!r/rs!%%s%7H7M%r1c "VP4pVPVP RV 2V4GRjxL #L \dp\ \ T44hRp?ii;i5i)a Check the status of an asynchronous extraction job. Args: job_id (str): The ID of the extraction job Returns: V1ExtractResponse[Any] with: * success (bool): Whether request succeeded * data (Optional[Any]): Extracted data matching schema * error (Optional[str]): Error message if any * warning (Optional[str]): Warning message if any * sources (Optional[List[str]]): Source URLs if requested Raises: ValueError: If status check fails rNrrrrrr,)rrrtrgs&& r!r&AsyncV1FirecrawlApp.get_extract_statussl$'') %00<<. VH5  %SV$ $ %1A&$?=?A&? A# AA##A&c<V^8dQhRS[S[S[,,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[S[S[3,,R S[S[,/ #rr)r-r.s"r!r/r,sD%D%49%D%SM D% SM D% $C= D%#+4.D% (~D%#4.D%DcN+D%8I7MD%r1c "VP4p V'gV'g \R4hV'gV'g \R4hV'dVPV4p\T;'g.VVVVR\ 2R7p V'dW*R&V'dWJR&V'dWR&VP VP R2V V 4GR jxL #L \dp \\T 44hR p ? ii;i5i) aV Initiate an asynchronous extraction job without waiting for completion. Args: urls (Optional[List[str]]): URLs to extract from prompt (Optional[str]): Custom extraction prompt schema (Optional[Any]): JSON schema/Pydantic model system_prompt (Optional[str]): System context allow_external_links (Optional[bool]): Follow external links enable_web_search (Optional[bool]): Enable web search show_sources (Optional[bool]): Include source URLs agent (Optional[Dict[str, Any]]): Agent configuration idempotency_key (Optional[str]): Unique key to prevent duplicate requests Returns: V1ExtractResponse[Any] with: * success (bool): Whether request succeeded * data (Optional[Any]): Extracted data matching schema * error (Optional[str]): Error message if any Raises: ValueError: If job initiation fails rrr)rr"rNrPr}rOr*rrrN) rrrrSrr rrr,) rrr*r}rrrrrrtrrgs &&$$$$$$$ r!r!AsyncV1FirecrawlApp.async_extract,sD'')fBC CF@A A --f5F(3-$  *   %+ " +8 ( $) ! %11<<. ,   %SV$ $ %sh C:C:C:C:C:0C: C: C:$C:)#C C CC:C C7C22C77C:rrrc h<V^8dQhRS[RS[S[,RS[S[,RS[S[,RS[/#)r(r^rrrrr)r-r.s"r!r/rrsQ;W;W;Wsm ;W %TN ;W "*$ ;W <\ ;Wr1c "/pVeW%R&VeW5R&VeWER&VPVVVVR7GRjxL pVPR4'dRV9dV#VR,pVPV4GRjxL pVR,R 8XdV#VR,R 8Xd\R VPR 4 24hVR,R 8wdM \P !^4GRjxL K\ RRR RR7#LLL5i)aw Generate LLMs.txt for a given URL and monitor until completion. Args: url (str): Target URL to generate LLMs.txt from max_urls (Optional[int]): Maximum URLs to process (default: 10) show_full_text (Optional[bool]): Include full text in output (default: False) experimental_stream (Optional[bool]): Enable experimental streaming Returns: V1GenerateLLMsTextStatusResponse containing: * success (bool): Whether generation completed successfully * status (str): Status of generation (processing/completed/failed) * data (Dict[str, str], optional): Generated text with fields: - llmstxt (str): Generated LLMs.txt content - llmsfulltxt (str, optional): Full version if requested * error (str, optional): Error message if generation failed * expiresAt (str): When the generated data expires Raises: Exception: If generation fails Nrrr)rrrrrr rvrwz#LLMs.txt generation failed. Error: rrWFrr0r)rrrrrrr) rr^rrrrr rr s &&$$$ r!r&AsyncV1FirecrawlApp.generate_llms_textrs*:   (9   %%3> "  *.A* +66 ) 3 7   ||I&&$h*>O$??GGFh;. !X-"EfjjQXFYEZ [\\!\1--" " "/Ev@HTVW W- H #s:3C<C6C<'C<8C89A'C< C:!C<8C<:C<rc ~<V^8dQhRS[RS[S[,RS[S[,RS[S[,RS[S[,RS[/#rr)r-r.s"r!r/rsX5%5%5%sm 5% %TN 5% D> 5%"*$5% "  *.A* +)'"5  '')CQ6;;4;#PQ +G95 ( %11<<. ,   %SV$ $ %s<AB1 #B BB B1B B.B))B..B1c&<V^8dQhRS[RS[/#rWr)r-r.s"r!r/rs%%%@`%r1c "VP4pVPVP RV 2V4GRjxL #L \dp\ \ T44hRp?ii;i5i)a Check the status of an asynchronous LLMs.txt generation job. Args: id (str): The ID of the generation job Returns: V1GenerateLLMsTextStatusResponse containing: * success (bool): Whether generation completed successfully * status (str): Status of generation (processing/completed/failed) * data (Dict[str, str], optional): Generated text with fields: - llmstxt (str): Generated LLMs.txt content - llmsfulltxt (str, optional): Full version if requested * error (str, optional): Error message if generation failed * expiresAt (str): When the generated data expires Raises: ValueError: If status check fails rNrerrrtrgs&& r!r3AsyncV1FirecrawlApp.check_generate_llms_text_statussl('') %00<<. RD1  %SV$ $ %rgr#r$/_AsyncV1FirecrawlApp__experimental_stream_stepsr&r'c.<V^8dQhRS[RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[S[S[S[3,.R 3,,R S[S[S[S[S[3,.R 3,,R S[/ #) r(r_r;r#rr$rrur&Nr'rr))r-r.s"r!r/rs^n^n^n } ^n ! ^n sm ^n&c]^n$C=^n*2$^n"(DcN+;T+A"BC^n $sCx.)94)? @A^nNj^nr1c F"/p VeW*R&VeW:R&VeWJR&VeWZR&VeWjR&VeWzR&\R/V Bp VPVVVVVVR7GRjxL p V PR 4'dR V 9dV #V R ,p ^p ^pVPV 4GRjxL pV'd7R V9d0VR ,V RpVF pV!V4K \ VR ,4p V 'd7R V9d0VR ,VRpVF pV !V4K \ VR ,4pVR ,R8XdV#VR ,R8Xd\ RVPR4 24hVR ,R8wdM!\ P!^4GRjxL EK\RRR7#EL@LL5i)r+Nrrrrrr,r-rrrrUr rvrwr.rrWFr/rr2) rr0rr1r_rrrr)rr_r;r#rr$rrur&r'r2r rr3r4r r5r6r7r8s&&$$$$$$$$ r!r9!AsyncV1FirecrawlApp.deep_researchsP  *3J '  !+5K (  )1I &  &0?, -  $.;N + & 2O$::6BBF|v5!' !56I6J!K .H)!/&)&*>&?#Y&0$Y/0A0BC )Ff%*$'y(9$:!h;. !X-"? 7@S?T UVV!\1--" " "+EAlmmO C* #sIAF!FF!6+F!!F" F!.=F!,BF!FF!F!F!c<V^8dQhRS[RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[,RS[S[S[3,/#) r(r_r;r#rr$rrurr<)r-r.s"r!r/rcs}:%:%:% } :% ! :% sm :%&c]:%$C=:%*2$:%DHS>:%r1c "/pVeW(R&VeW8R&VeWHR&VeWXR&VeWhR&VeWxR&\R/VBpVP4p RV/VPR R R 7Cp R \ 2V R &VP VP R 2V V 4GRjxL #L \ dp \\T 44hRp ? ii;i5i)r>Nrrrrrr,r_TrrrOr?r2) rrrrr rrrr,) rr_r;r#rr$rrur2rtrrgs &&$$$$$$ r!r0'AsyncV1FirecrawlApp.async_deep_researchcs>  *3J '  !+5K (  )1I &  &0?, -  $.;N + & 2>#ABCN *N r1c  B"/p V 'dK\V \4'dV PV 4M"V PV PRRR74VeW,R&VeWuB48uB 37!uB"-1#uB$ $%uB&,0'uB(())uB**.+uBuBpsB.2 sB .2 sB $( sB.2sB $sB04sB/3sB04sB*.sB59sB:>sB48sB 37!sB"-1#sB$ $%sB&,0'sB(())sB**.+sBsBj@@D)_)_VKE!% KE *. KE .2 KE(,KE $KE"'KE04KE)-KEKEZdR%) dR %) dR ,0 dR49dR16dR,1dR/3dRdRLB B H  2  2--(%%6D%%) D% %) D% ,0 D%49D%16D%,1D%/3D%D%L;W'+ ;W .2 ;W 37 ;W;Wz5%'+ 5% .2 5% %) 5%375%5%n%%:^n(, ^n )- ^n '+ ^n.2^n,0^n;?^nGK^nEI^n^n@:%(, :% )- :% '+ :%.2:%,0:%;?:%:%x % %DN $( N "& N %) N #'N &*N '+N &+N 9=N GKN N N r1rcaa]tRtRtoRtV3RlV3RlltV3RlRltV3RlRltV3R lR ltV3R lR lt V3R lRlt Rt Vt V;t #)r-izI Async version of V1CrawlWatcher that properly handles async operations. c&<V^8dQhRS[RS[/#r)r,r)r-r.s"r!r/ AsyncV1CrawlWatcher.__annotate__s""3"%8"r1c&<\SV`W4R#rr)rrrrs&&&r!rAsyncV1CrawlWatcher.__init__s !r1c<V^8dQhRR/#rr2)r-r.s"r!r/rs**t*r1c @"\P!VPRRVPP 23.R7;_uu_4GRjxL pVP V4GRjxL RRR4GRjxL R#L/LL +GRjxL 'giR#;i5i)zK Establishes async WebSocket connection and starts listening for messages. rr)rNrrs& r!rAsyncV1CrawlWatcher.connects%% KK!0GDHH<V^8dQhRS[PRS[RR/#rr)r-r.s"r!r/rCs)++G,B,B+C+TX+r1c f"VP4GRjxL pVPRR4pVPRR4pTP P TPY$T4GRjxL p\P!T4hLm \P!RTP 24h;iLD5i)z) Handle errors from async API responses. Nrrrrr)rOrrrr rrrs&&& r!r!AsyncV1CrawlWatcher._handle_errorCs {'}}.J&NN74PQM&NN96]^M 99(//6bopp!!'**/ {%%(ghphwhwgx&yz zqs8B1BB(B*B1*B/+B1B%B,,B1c 8<V^8dQhRS[RS[RS[RS[RS[/#rr)r-r.s"r!r/rSrr1c ."VPWW44#5irrr!s&&&&&r!r,AsyncV1CrawlWatcher._get_async_error_messageSr!r")r )r3r4r5r6r7rrrrrrr9r:rrs@@r!r-r-sP""** , ,RR,++ Z Z Zr1r-cTV^8dQh/^\9d\P;R&#)r(r)__conditional_annotations__loggingLogger)r-s"r!r/r/s"  49885 r1)Trr7rrrtypingrrrrrrrr r rOr rrrYrrr getLoggerrr"rr# BaseModelr%r=rGrNr\rlrrr|rrrrrrrrrrrrrrrr rr-r1r5r;rArHrLrSrZrerjrsrzrrrrrrrrrrrrrr{rr-r/)rs@r!rsV  XXX  !++K88  - CLJ!X''! (H..(h(( C8--C:(,,gaj: *x))* Uh((Uh00$h(($(#8%%# "++" H&& H&&  H&& #X''# X'' 2 2 "($$"(X''(+8%%+._. *1-wqz ,H.., $("4"4 $+H&&+( h((  $H.. $8%%H.. 0($$ 0 H&& 4h(( 4 &**GAJ & 4X'' 4 x)) -**- H..-x))- 8--H$6$6 1h&8&818#5#5 0X%7%701x111 68-- 6 X//  8#5#5  !3!3 &8+=+=&x'9'9 x))  +h(( +b* b* JUYRYRvh .h T9OZ.OZr1