9h^ SSKrSSKrSSKJr SSKJr SSKJr SSKJ r SSK J r J r SSK JrJrJrJrJr SSKJrJrJrJr SS KJr S S KJrJrJrJrJrJ r SS K!J"r"J#r# S S K$J%r% S SK&J'r'J(r(J)r)J*r*J+r+J,r,J-r- \S\.\/\0\Rb\2\3\4\54 r6\\.\4\54r7"SS\5r8S\.S\4Sjr9S\S\ S\\.\744Sjr:"SS\'5r;"SS\(5r<"SS\Rz5r>S\.S\\.\74S\74Sjr?S\)S\7S \6S!\+S\64 S"jr@S\)S\7S#\5S \6S!\+4 S$jrAS\7S\*4S%jrBS&\\.S'\\.S\24S(jrCS\7S \6S\\74S)jrDS\7S\\.\\.44S*jrES+\.S,\.S\\7S-\\.\\.44S.jrFS,\.S\\.4S/jrGg)0N) defaultdict)deepcopy)BytesIO)loads)packunpack)DictUnionOptionalSetCallable)schemaless_readerschemaless_writer repositoryvalidate) load_schema) _MAGIC_BYTESchematopic_subject_name_strategyRuleModeRuleKindSchemaRegistryClient)SerializationErrorSerializationContext) RuleRegistry)BaseSerializerBaseDeserializer RuleContext FieldTypeFieldTransformRuleConditionErrorParsedSchemaCachec$\rSrSrSrSrSrSrg)_ContextStringIO;z9 Wrapper to allow use of StringIO via 'with' constructs. cU$N)selfs iC:\Suresh\moveshuttle\MDcreated\moveengine\venv\Lib\site-packages\confluent_kafka/schema_registry/avro.py __enter___ContextStringIO.__enter__@s c$UR5 g)NF)close)r*argss r+__exit___ContextStringIO.__exit__Cs r.r)N)__name__ __module__ __qualname____firstlineno____doc__r,r2__static_attributes__r)r.r+r%r%;sr.r% schema_strreturncjUR5nUSS:waUSS:waSU-S-n[USS9$)z Instantiate a Schema instance from a declaration string. Args: schema_str (str): Avro Schema declaration. .. _Schema declaration: https://avro.apache.org/docs/current/spec.html#schemas Returns: Schema: A Schema instance. r{[z{"type":}AVRO) schema_type)stripr)r:s r+ _schema_loadsrCHsG!!#J!} 1  4*,s2 *& 11r.schemaschema_registry_clientc>0nURbURH}nURURURS5n[ UR U5n[ UR RUS9nURU5 XbUR'M U$)z Resolves named schemas referenced by the provided schema recursively. :param schema: Schema to resolve named schemas for. :param schema_registry_client: SchemaRegistryClient to use for retrieval. :return: named_schemas dict. T named_schemas) references get_versionsubjectversion_resolve_named_schemarDparse_schema_with_repor:updatename)rDrErHrefreferenced_schemaref_named_schemas parsed_schemas r+rMrM_sM $$$C 6 B B3;;PSP[P[]a b  56G6N6NPf g 2!((33CTVM  !2 3&3#(( # % r.c^\rSrSrSr/SQrSSSSS\S.rSS\S \ \ \ S4S \ \ \\/\4S \ \S \ \S \ \4 U4SjjjrSS\S\ \S\ \4SjjrS\ S\4SjrSrU=r$)AvroSerializertaA Serializer that outputs Avro binary encoded data with Confluent Schema Registry framing. Configuration properties: +-----------------------------+----------+--------------------------------------------------+ | Property Name | Type | Description | +=============================+==========+==================================================+ | | | If True, automatically register the configured | | ``auto.register.schemas`` | bool | schema with Confluent Schema Registry if it has | | | | not previously been associated with the relevant | | | | subject (determined via subject.name.strategy). | | | | | | | | Defaults to True. | +-----------------------------+----------+--------------------------------------------------+ | | | Whether to normalize schemas, which will | | ``normalize.schemas`` | bool | transform schemas to have a consistent format, | | | | including ordering properties and references. | +-----------------------------+----------+--------------------------------------------------+ | | | Whether to use the given schema ID for | | ``use.schema.id`` | int | serialization. | | | | | +-----------------------------+----------+--------------------------------------------------+ | | | Whether to use the latest subject version for | | ``use.latest.version`` | bool | serialization. | | | | | | | | WARNING: There is no check that the latest | | | | schema is backwards compatible with the object | | | | being serialized. | | | | | | | | Defaults to False. | +-----------------------------+----------+--------------------------------------------------+ | | | Whether to use the latest subject version with | | ``use.latest.with.metadata``| dict | the given metadata. | | | | | | | | WARNING: There is no check that the latest | | | | schema is backwards compatible with the object | | | | being serialized. | | | | | | | | Defaults to None. | +-----------------------------+----------+--------------------------------------------------+ | | | Callable(SerializationContext, str) -> str | | | | | | ``subject.name.strategy`` | callable | Defines how Schema Registry subject names are | | | | constructed. Standard naming strategies are | | | | defined in the confluent_kafka.schema_registry | | | | namespace. | | | | | | | | Defaults to topic_subject_name_strategy. | +-----------------------------+----------+--------------------------------------------------+ Schemas are registered against subject names in Confluent Schema Registry that define a scope in which the schemas can be evolved. By default, the subject name is formed by concatenating the topic name with the message field (key or value) separated by a hyphen. i.e. {topic name}-{message field} Alternative naming strategies may be configured with the property ``subject.name.strategy``. Supported subject name strategies: +--------------------------------------+------------------------------+ | Subject Name Strategy | Output Format | +======================================+==============================+ | topic_subject_name_strategy(default) | {topic name}-{message field} | +--------------------------------------+------------------------------+ | topic_record_subject_name_strategy | {topic name}-{record name} | +--------------------------------------+------------------------------+ | record_subject_name_strategy | {record name} | +--------------------------------------+------------------------------+ See `Subject name strategy `_ for additional details. Note: Prior to serialization, all values must first be converted to a dict instance. This may handled manually prior to calling :py:func:`Producer.produce()` or by registering a `to_dict` callable with AvroSerializer. See ``avro_producer.py`` in the examples directory for example usage. Note: Tuple notation can be used to determine which branch of an ambiguous union to take. See `fastavro notation `_ Args: schema_registry_client (SchemaRegistryClient): Schema Registry client instance. schema_str (str or Schema): Avro `Schema Declaration. `_ Accepts either a string or a :py:class:`Schema` instance. Note that string definitions cannot reference other schemas. For referencing other schemas, use a :py:class:`Schema` instance. to_dict (callable, optional): Callable(object, SerializationContext) -> dict. Converts object to a dict. conf (dict): AvroSerializer configuration. )_known_subjects_parsed_schema_schema _schema_id _schema_name_to_dict_parsed_schemasTFN)auto.register.schemasnormalize.schemas use.schema.iduse.latest.versionuse.latest.with.metadatasubject.name.strategyrEr:to_dictconf rule_conf rule_registryc>[T U]5 [U[5(a [ U5nO[U[ 5(aUnOSnXlSUlU(aUO[R"5Ul [5Ul [5UlUb[U5(d [!S5eX0lUR$R'5nUbUR)U5 UR+S5Ul[UR,[.5(d [!S5eUR+S5Ul[UR0[.5(d [!S5eUR+S5UlUR2b*[UR2[45(d [!S5eUR+S5Ul[UR6[.5(d [!S 5eUR6(aUR,(a [!S 5eUR+S 5UlUR8b*[UR8[:5(d [!S 5eUR+S 5Ul[UR<5(d [!S5e[?U5S:a7[!SRASRCURE5555eU(a`URGU5n [U [H5(aSn O;[KURL5n U ROSU ROS55n OSn Sn Xpl(Xl)Xl*URRW5HIn U RYUR (aUR R[5O0U(aUO05 MK g)NzWto_dict must be callable with the signature to_dict(object, SerializationContext)->dictr_z-auto.register.schemas must be a boolean valuer`z)normalize.schemas must be a boolean valueraz"use.schema.id must be an int valuerb*use.latest.version must be a boolean valuez?cannot enable both use.latest.version and auto.register.schemasrc-use.latest.with.metadata must be a dict valuerd&subject.name.strategy must be callablerUnrecognized properties: {}, rPtype).super__init__ isinstancestrrCr _registryr[rget_global_instance_rule_registrysetrXr#r^callable ValueErrorr] _default_confcopyrOpop_auto_registerbool_normalize_schemas_use_schema_idint_use_latest_version_use_latest_with_metadatadict_subject_name_funclenformatjoinkeys_get_parsed_schemalistrr:getrZr\rY get_executors configureconfig)r*rEr:rerfrgrhrD conf_copyrT schema_name schema_dictrule __class__s r+rqAvroSerializer.__init__s   j# & &":.F  F + +FF//A :$fTYYy~~/?%@AC C  33F;M-.. # $F$5$56 +// 8OP K M '+''557D NNdnn4>>002"(19r ;8r.objctxr;c ^ UcgURX R5nURU5nUbURUlOX0R ;aUR (a6URRX0RUR5UlOAURRX0RUR5nURUlUR RU5 URbURX5nOUnUbZURUR5m U 4SjnUR!X#["R$SURU['T 5U5nO UR(m [+5nUR-[/S[0UR55 [3UT U5 UR55sSSS5 $!,(df  g=f)aK Serializes an object to Avro binary format, prepending it with Confluent Schema Registry framing. Args: obj (object): The object instance to serialize. ctx (SerializationContext): Metadata pertaining to the serialization operation. Raises: SerializerError: If any error occurs serializing obj. SchemaRegistryError: If there was an error registering the schema with Schema Registry, or auto.register.schemas is false and the schema was not registered. Returns: bytes: Confluent Schema Registry encoded Avro bytes Nc>[UTX!5$r( transform)rule_ctxfield_transformmsgrTs r+)AvroSerializer.__call__..ss(M3Hr.>bI)rr\_get_reader_schema schema_idr[rXr}rtregister_schemarZr lookup_schemaaddr]rrD_execute_rulesrWRITEget_inline_tagsrYr%writerrrgetvalue) r*rrrK latest_schemaregistered_schemavaluefield_transformerforTs @r+__call__AvroSerializer.__call__Bs( ;))#/@/@A//8  $+55DO 00 0""#'.."@"@\\4+B+B#D%)NN$@$@\\4+B+B%D!"3"="=  $ $W - == $MM#+EE  $ 33M4H4HIM!J ''hnnd(5(<( ? b- 7;;=   s AG G,rDcURRU5nUbU$[XR5n[ UR 5n[ UR US9nURRX5 U$NrGr^get_parsed_schemarMrtrCr:rNrwr*rDrTrHprepared_schemas r+r!AvroSerializer._get_parsed_schemat,,>>vF  $ -fnnE '(9(9:.  & &mE    7r.)r}rXrrYr^rtrvrZr[r\rr]rrr)NNNNNr()r4r5r6r7r8 __slots__rrzrr rsrr r objectrrrrqbytesr AvroSchemarr9 __classcell__rs@r+rVrVtsdJNI/3*/&*+015.I KM04LP#$(04\; 4\;#vt+,\;(F,@#A4#GHI \; tn \; D> \; -\;\;|?!F?!2F)G?!S[\aSb?!B  J  r.rVc^\rSrSrSr/SQrSS\S.rSS\S\ \ \ S4S \ \ \\/\4S \S \ \S \ \S \ \4U4SjjjrSS\S\ \S\ \\S44SjjrS\ S\4SjrSrU=r$)AvroDeserializeria Deserializer for Avro binary encoded data with Confluent Schema Registry framing. +-----------------------------+----------+--------------------------------------------------+ | Property Name | Type | Description | +-----------------------------+----------+--------------------------------------------------+ | | | Whether to use the latest subject version for | | ``use.latest.version`` | bool | deserialization. | | | | | | | | Defaults to False. | +-----------------------------+----------+--------------------------------------------------+ | | | Whether to use the latest subject version with | | ``use.latest.with.metadata``| dict | the given metadata. | | | | | | | | Defaults to None. | +-----------------------------+----------+--------------------------------------------------+ | | | Callable(SerializationContext, str) -> str | | | | | | ``subject.name.strategy`` | callable | Defines how Schema Registry subject names are | | | | constructed. Standard naming strategies are | | | | defined in the confluent_kafka.schema_registry | | | | namespace. | | | | | | | | Defaults to topic_subject_name_strategy. | +-----------------------------+----------+--------------------------------------------------+ Note: By default, Avro complex types are returned as dicts. This behavior can be overridden by registering a callable ``from_dict`` with the deserializer to convert the dicts to the desired type. See ``avro_consumer.py`` in the examples directory in the examples directory for example usage. Args: schema_registry_client (SchemaRegistryClient): Confluent Schema Registry client instance. schema_str (str, Schema, optional): Avro reader schema declaration Accepts either a string or a :py:class:`Schema` instance. If not provided, the writer schema will be used as the reader schema. Note that string definitions cannot reference other schemas. For referencing other schemas, use a :py:class:`Schema` instance. from_dict (callable, optional): Callable(dict, SerializationContext) -> object. Converts a dict to an instance of some object. return_record_name (bool): If True, when reading a union of records, the result will be a tuple where the first value is the name of the record and the second value is the record itself. Defaults to False. See Also: `Apache Avro Schema Declaration `_ `Apache Avro Schema Resolution `_ )_reader_schema _from_dict_return_record_namerZr^FN)rbrcrdrEr: from_dictreturn_record_namerfrgrhc>[T U]5 SnUbD[U[5(a [ U5nO#[U[ 5(aUnO [ S5eXlXlU(aUO[R"5Ul [5Ul SUlURR!5n UbU R#U5 U R%S5Ul[UR&[(5(d [+S5eU R%S5UlUR,b*[UR,[.5(d [+S5eU R%S5Ul[3UR05(d [+S5e[5U 5S:a7[+S R7S R9U R;5555eU(a!UR=UR5UlOSUlUb5(d [+S 5eX0l X@l![URB[(5(d [+S 5eURRE5HIn U RGUR(aURRI5O0U(aUO05 MK g) Nz3You must pass either schema string or schema objectrbrjrcrkrdrlrrmrnz]from_dict must be callable with the signature from_dict(SerializationContext, dict) -> objectz*return_record_name must be a boolean value)%rprqrrrsrCr TypeErrorrZrtrrurvr#r^rrzr{rOr|rr~ryrrrrxrrrrrrrrrrr) r*rEr:rrrfrgrhrDrrrs r+rqAvroDeserializer.__init__s5   !*c**&z2J//# UVV //A :$fTYYy~~/?%@AC C "&"9"9$,,"GD "&D   ))<)<OP P##5 $22D99IJ J''557D NNdnn4>>002"(19r ;8r.datarr;c x^Ucg[U5S::a#[SR[U555eURUS5nSnUbUR U5n[ U5n[ SURS55upgU[:wa[SRU55eURRU5nURU5n Uc4URX)RS55nUbUR U5nUb;URX8US5n URn URUR5mO.URbSn URn UR mOSn Un U mU (a+[#UU SUR$5n UR'X#X5n O[#UU TUR$5n U4Sjn UR)X#[*R,SX[/T5U 5n UR0bUR1X5sSSS5 $U sSSS5 $!,(df  g=f)a Deserialize Avro binary encoded data with Confluent Schema Registry framing to a dict, or object instance according to from_dict, if specified. Arguments: data (bytes): bytes ctx (SerializationContext): Metadata relevant to the serialization operation. Raises: SerializerError: if an error occurs parsing data. Returns: object: If data is None, then None. Else, a dict, or object instance according to from_dict, if specified. NzExpecting data framing of length 6 bytes or more but total data size is {} bytes. This message was not produced with a Confluent Schema Registry serializerrzcUnexpected magic byte {}. This message was not produced with a Confluent Schema Registry serializerrPc>[UTX!5$r(r)rrmessage reader_schemas r+r+AvroDeserializer.__call__..]s(M7Lr.)rrrrrr%rreadrrt get_schemarr_get_migrationsrDrZrrr_execute_migrationsrrREADrr)r*rrrKrpayloadmagicrwriter_schema_raw writer_schema migrationsreader_schema_rawobj_dictrrs @r+rAvroDeserializer.__call__s8& < t9>$&BCI&TBSU U ))#t4   33GH*x5_$ #bc$ # #s+F+H+ H++ H9rDcURRU5nUbU$[XR5n[ UR 5n[ UR US9nURRX5 U$rrrs r+r#AvroDeserializer._get_parsed_schemahrr.) rr^rrtrrvrZrrrr)NNFNNNr()r4r5r6r7r8rrrzrr rsrr r rrrr~rrqrrrrr9rrs@r+rrs8t/I,115.IKM04NR#(#$(04?; 4?;#vt+,?;Hd,@%A6%IJK ?; ! ?; tn ?;D>?; -?;?;BRUR2F)GRSXY]_egkYkSlRh  J  r.rc \rSrSrSrSrSrg)LocalSchemaRepositoryivcXlgr(schemas)r*rs r+rqLocalSchemaRepository.__init__ws r.c8URRU5$r()rr)r*rKs r+loadLocalSchemaRepository.loadzs||((r.rN)r4r5r6r7rqrr9r)r.r+rrvs )r.rrHc^[U5n[U5US'[U5n[SUS9$)Nz$root)repo)rrrr)r:rHr{rs r+rNrN~s1 M "D*%DM  &D wT **r.rrrc UbUcU$UR5nUb[U5Ul[U[5(a[ X5nUcU$[ XX#5$[U[5(aURS5nUS:Xa UVs/sHn[ XSXs5PM sn$US:Xa3UR5VV s0sHupU[ XSX5_M sn n$US:XaUSn U Hn [XXU5 M U$UbJURRn U (a$[[U 5UR5(d U"XU5$U$s snfs sn nf)Nroarrayitemsmapvaluesrecordfields) current_fieldget_type field_typerrr_resolve_unionrrrr_transform_fieldrtags _disjointrw) rrDrr field_ctx subschemarAitemkeyrrfield rule_tagss r+rrsd&.!!#I'/ &$"63  NBB FD ! !jj( ' ! ') 'c'?DJ ') ) E !&-mmo7&5 3x(8%QQ&57 7 H $H%F eoN NHHMM  #i.).. I I"37; ; N)7s EErcdUSnUSnUSS-U-nURUUU[U5S5 X6n[XX5n URR[ R :XaU SLa[UR5eOXU'UR5 g!UR5 f=f)NrorP.F) enter_fieldrrrkindr CONDITIONr" exit_field) rrDrrrrrP full_namer new_values r+rrsvJ =Dv$t+I     Z    cuF 88==H.. .E!(22"&DM s A5BB/c&[U[5(a[R$[U[5(aUR S5nOUnUS:Xa[R $US:Xa[R$US:Xa[R$US:Xa[R$US:Xa[R$US:Xa[R$US:Xa[R$US :Xa[R$US :Xa[R$US :Xa[R$US :Xa[R$US :Xa[R $US:Xa[R"$US:Xa[R$$[R$$)Nrorenumrrunionfixedstringrrlongfloatdoublebooleannull)rrrr COMBINEDrrRECORDENUMARRAYMAPFIXEDSTRINGBYTESINTLONGFLOATDOUBLEBOOLEANNULL)rDrAs r+rrsM&$!!! FD ! !jj(  h  ~~    }}  !!!        }}  ~~      !    ~~~~r.tags1tags2c$UH nX!;dM g g)NFTr))r r!tags r+rrs < r.cBUHn[X5 Us $ g! M=fr()r)rDrrs r+rrs0   W (    s c>[[5n[SSX5 U$)N)rrw_get_inline_tags_recursively)rD inline_tagss r+rrsc"K R= r.nsrPrcUcg[U[5(aUHn[XXC5 M g[U[5(dgUR S5nUS:Xa[XUR S5U5 gUS:Xa[XUR S5U5 gUS:XaUR S5nUR S5nUc [ U5nUcUnUS :waUR U5(dUS U3nUS nUHfn U R S 5n U R S5n U R S5n U bU bX7S -U -RU 5 U cMZ[XgX5 Mh gg) Nrorrrrr namespacerPr&rrzconfluent:tags)rrrr'rr_implied_namespace startswithrO) r)rPrDrrrA record_ns record_namerr field_tags field_namers r+r'r'se~&$I (9 C  % %jj( ' ! (6::g3F M E ! (6::h3G N H $ ;/I **V,K .t4   B{'='=i'H'H!* 1[M: H%F"YY'78 "YYv. "YYv. )j.Ds*Z78?? K)0Z %r.cd[R"SU5nU(aURS5$S$)Nz ^(.*)\.[^.]+$r)rematchgroup)rPr4s r+r,r,(s( HH%t ,E"5;;q>,,r.)Hdecimalr3 collectionsrr{riorjsonrstructrrtypingr r r r r fastavrorrrrfastavro.schemarr&rrrrrrconfluent_kafka.serializationrrrhrserderrrr r!r"r#rsrrDecimalr~rrr AvroMessagerr%rCrMrVrAbstractSchemaRepositoryrrNrrrrrrr'r,r)r.r+rCsU$ #77  (// A':::  OO    3d? #  w 2c2f2. ,@ #z/*Z^Zzb'bJ)J??)+s+4Z;P+U_+   ( 3> #  F (15 +94&Z&I&RSXc#h4: @TJ4SX +> #[ #[#[ ( 4#[ sCH} #[L-S-Xc]-r.