@j(gdZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddlZ ddlZddlZddlZejeZGddeZGddZdZdd ZejGd d ZejGd d eZGddeZdZ e Z!Gdde j"j#Z$Gddej%j&Z'Gdde(Z)Gdde)Z*GddZ+e j,-ee\Z.Z/dS))DeprecatedFeatureWarningItemIntWithGranularityScraperExceptionEntityUnavailableScraperNceZdZdS)rN__name__ __module__ __qualname__T/home/ubuntu/.hermes/hermes-agent/venv/lib/python3.11/site-packages/snscrape/base.pyrrrrceZdZdZdZdS)_DeprecatedPropertyc0||_||_||_dSN)namereplreplStr)selfrrrs r__init__z_DeprecatedProperty.__init__s$)$)$,,,rc||Stj|jd|jdtd||S)Nz is deprecated, use z instead) stacklevel)warningswarnrrrr)robjobjTypes r__get__z_DeprecatedProperty.__get__#sJ[ ; -49HH$,HHHJbqrssss 3rN)r r r rr"rrrrrs2 rrct|tjtjfr|St|tjr|jStdt|d)zA JSON serialiser that converts datetime.datetime and datetime.date objects to ISO-8601 strings and enum.Enum objects to their values.zObject of type z is not JSON serializable) isinstancedatetimedate isoformatenumEnumvalue TypeErrortype)r s r_json_serialise_datetime_enumr-*sjsX& 677 sDI G499GGGHHHrFct|tstj|r&i}t |jdt |j|d<tj|D]V}|jdksJ|j dr*tt||j||j<Wt|D]z}ttt ||dttfr@|dksJ| drXtt||||<{nt|tt fr$t |fd|DSt|t"r!fd|D}n7t|t&rfd|DSt)j|St!|D]\}}t|t,r5t/|||<|d|vsJd |d|j||d<OrRt|t.r=t3|d kr*|d |vsJd |d t5|||d <|S) N._type_forBuggyIntParserc3:K|]}t|VdS)r2N_json_dataclass_to_dict).0xr3s r z*_json_dataclass_to_dict..Es3bbYZ*1BSTTTbbbbbbrcXi|]&\}}t|t|'Sr2r5)r7kvr3s r z+_json_dataclass_to_dict..Gso g g gSWSTVW 8I J J JLcde|MMNMNMN g g grc2h|]}t|Sr;r5)r7r=r3s r z*_json_dataclass_to_dict..Is) Y Y YPQ !!9J K K K Y Y Yrz .granularityzGranularity collision on lz.strzBuggy int collision on )r$_JSONDataclass dataclasses is_dataclassr,r r fieldsr startswithr6getattrdirpropertyrtuplelistdictitemssetcopydeepcopyrint granularityabsstr)r r3outfieldr<keyr*s ` rr6r64s2sN##{'?'D'D #II(??499+=??#g,!#&&nne *     jC    ,WS%*-E-E[lmmm3uz?? s88]]acAt,,x9L.MNN] <<<<||C $WS!__J[ \ \ \CF ] udm$$ cbbbb^abbb b bbd g g g g[^[d[d[f[f g g g##c Y Y Y YUX Y Y YY s   $$""ZS%)**"%jj3s8    c ) ) )+Xs+X+X+X ) ) )$03#"Zs33"E U8J8J ,,,c ! ! !#FS#F#F#F ! ! !5zz3#||| rceZdZdZddZdS)rAz3A base class for dataclasses for conversion to JSONFc*tj5tjdtt ||}dddn #1swxYwYd|vs Jdt jj|d<tj |tS)a` Convert the object to a JSON string If forBuggyIntParser is True, emit JSON for parsers that can't correctly decode integers exceeding the limits of double-precision IEEE 754 floating point numbers. Specifically, each field x containing an integer with a magnitude above 2**53 results in an additional field x.str with the value as a string. ignore)actioncategoryr2N _snscrapezMetadata collision on _snscrape)default) rcatch_warningsfilterwarningsrr6snscrapeversion __version__jsondumpsr-)rr3rTs rrcz_JSONDataclass.json\s  NN H9QRRRR ;L M M M3NNNNNNNNNNNNNNN C   !B   %1#k C#@ A A AAs-A  AANF)r r r __doc__rcrrrrArAXs4:: B B B B B BrrAc2eZdZdZejdZdS)rzAn abstract base class for an item returned by the scraper. An item can really be anything. The string representation should be useful for the CLI output (e.g. a direct URL for the item). cdSrrrs r__str__z Item.__str__s$rN)r r r rfabcabstractmethodrjrrrrrls= rrc(eZdZdZfdZdZxZS)rzA number with an associated granularity For example, an IntWithGranularity(42000, 1000) represents a number on the order of 42000 with two significant digits, i.e. something counted with a granularity of 1000. cRtj||g|Ri|}||_|Sr)super__new__rQ)clsr*rQargskwargsr __class__s rrqzIntWithGranularity.__new__~s6U4T444V44##/ *rc<tt||jffSr)rrPrQris r __reduce__zIntWithGranularity.__reduce__s s4yy$*:; <.lerps# b&R"W b )B ..rii oezXMozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/z.0.0.0 Safari/537.36)rPr%r& toordinaltodayrandomrandintmax)rras r_random_user_agentrs/// ttHM$1--77998=qRT;U;U;_;_;a;acfhkmumznAnAnCnCnMnMnOnOPP Q QFN2q ! !! w   Acj A A AArceZdZfdZxZS) _HTTPSAdapterc @tj|i| t|jjd_dS#t tf$rR}t dt|j dt|j d|Yd}~dSd}~wwxYw)Nhttpsz%Could not install TLS cipher logger: r/ ) rpinit_poolmanager_HTTPSConnection poolmanagerpool_classes_by_scheme ConnectionClsAttributeErrorKeyError_loggerdebugr,r r )rrsrterus rrz_HTTPSAdapter.init_poolmanagers%''D+F+++hDT4*73AAA ( #hhh ==fa9KffdSTggN^ffabffggggggggghs:B ABB)r r r rrxrys@rrrsAhhhhhhhhhrrceZdZfdZxZS)rcRtj|i|} td|jn#t $rYnwxYw td|jn#t $rYnwxYw|S)NzConnected to: zConnection cipher: )rpconnectrrsock getpeernamercipher)rrsrtconnrus rrz_HTTPSConnection.connects $ )& ) )$ ==;$)"7"7"9"9;;<<<< 4 ==;ty'7'7'9'9;;<<<< 4 +s#4A AA"4B B$#B$)r r r rrxrys@rrrs8         rrceZdZdS)rNr rrrrrrrrceZdZdZdS)rzdThe target entity of the scrape is unavailable, possibly because it does not exist or was suspended.N)r r r rfrrrrrskkkkrrceZdZdZdZddddZejdZdZ e j dZ dd Z d Zd ZedZedZedZdS)rz%An abstract base class for a scraper.Nr)retriesproxiesc||_||_tj|_|jdt dS)Nzhttps://)_retries_proxiesrequestsSession_sessionmountr)rrrs rrzScraper.__init__sB$-$-"$$$--j-//22222rcdS)zIterator yielding Items.Nrris r get_itemszScraper.get_itemss $rcdS)zGet the entity behind the scraper, if any. This is the method implemented by subclasses for doing the actual retrieval/entity object creation. For accessing the scraper's entity, use the entity property. Nrris r _get_entityzScraper._get_entitys rc*|Sr)rris rentityzScraper.entitys    r Tc |si}d|vr t|d<| p|jpi} g} t|jdzD]e} |jt j|||||} |j| j | ddd} t d| j t d||rt d|| rt d|  |jj | f||d| }|jr d |j d nd }t d | j |d |jt d|j|jrt#|jD]u\}}t d|d |jj d |jd|jdd t d|jv|$||\}}| |nd\}}|rd |nd }|r(t | j d||cS| |jkrd}t*j}nd }t*j}t|d| j ||n#t jj$r|}| |jkrd}t*j}nd }t*j}t|d| j d ||| t7|Yd}~nd}~wwxYw| |jkr;dd| zz}t d|ddt9j|g|jdzd| j d}t|tdd| tA|) Nz User-Agentr)paramsdataheadersz Retrieving z... with headers: z... with data: z... with environmentSettings: )allow_redirectstimeoutz (redirected to )z Retrieved z: z... with response headers: z ... request z (Location: Locationz... ... with response headers: )TNz retrieved successfullyz , retryingzError retrieving g?rzWaiting z.0fz secondsz requests to z failed, giving up.zErrors: z, )"_DEFAULT_USER_AGENTrrangerrprepare_requestrRequestmerge_environment_settingsurlrinforsendhistory status_coder enumeraterequestgetappendloggingINFOERRORlog exceptionsRequestExceptionreprtimesleepfataljoinr RuntimeError)rmethodrrrrrresponseOkCallbackallowRedirectsrerrorsattemptreqenvironmentSettingsr redirectediredirectsuccessmsgretryinglevelexc sleepTimes r_requestzScraper._requests  7  .7<  *t} *' &t}q())77g  & &x'7f]amt'u'u'u v v3AA#'7TXZ^`dee <<'cg''((( ==1g11222 . MM,D,,---L MMJ3FJJKKK$F 3k.GkkWjkkA12 A,AE,,,,rJ LLDcgDzDDQ]DDEEE MM= ==>>>yL!!),,LLH mmF1FF(8(<FF@TFFbjbrbvbvxBcCcCFFFGGG mmJh6FJJKKKK%&&q))\Wc ]]3 \Wc #*s***C F ]]cg;;c;;<<< XXX$-hleehme [[DCGDSD(DDEEEEE   .X \UUX ]U KKG37GGcGXGGHHH MM$s))F aj I LL3I3333444Jy MA  H HCG H H H3 == ==/DIIf--//000 #  s KMA2MMc$|jdg|Ri|S)NGETrrrsrts r_getz Scraper._gets$ u .t . . .v . ..rc$|jdg|Ri|S)NPOSTrrs r_postz Scraper._posts$ v / / / / / //rcdSrr)rr subparsers r_cli_setup_parserzScraper._cli_setup_parserrkrc,||Sr)_cli_construct)rrrss r_cli_from_argszScraper._cli_from_argss  D ! !!rc$||i|d|jiS)Nr)r)rr argparseArgsrsrts rrzScraper._cli_construct s% d =f = = (< = = ==r)NNNrNTN)r r r rfrrrlrmrr functoolscached_propertyrrrr classmethodrrrrrrrrs,, !"d33333   ?1?1?1?1B///000+""+">>+>>>rr)Entityre)0__all__rlrNrBr%r(rrcrrrrequests.adapterssnscrape.utilsr`snscrape.versionurllib3.connectionurllib3rr getLoggerr r FutureWarningrrr-r6 dataclassrArrPrrradapters HTTPAdapterr connectionHTTPSConnectionr Exceptionrrrutilsmodule_deprecation_helper __getattr____dir__rrrrs$ x x x      ' H % %}        III! ! ! ! H BBBBBBBB& > = = = = = = = =AAA)(**hhhhhH%1hhhw)9"ylllll(llln>n>n>n>n>n>n>n>b ~??RV?WW WWWr