👨🔧
👨🔧
👨🔧
👨🔧
The Ops Compendium
Ask or search…
⌃K
Comment on page

Data Contract

  1. 1.
    A data contract is a formal agreement between a service and a client that abstractly describes the data to be exchanged. That is, to communicate, the client and the service do not have to share the same types, only the same data contracts. - Microsoft​
  2. 2.
    The following is a set of articles around the topics of data product / contract etc. however the focus is about data contracts and expectations IMO, therefore I place it here. - by Chad sanderson
    1. 1.
      ​the existential threat of data quality​
    2. 2.
      ​the death of data modeling part 1​
    3. 3.
      ​data's collaboration problem​
    4. 4.
      the rise of data contracts​
    5. 5.
      ​production grade data products​
    6. 6.
      (finally!) a guide to data contracts p1​
    7. 7.
      (good) contracts, robustness in datamesh (but not only), also has about data-sharing agreements.
    8. 8.
      (good) from zero to hero​
    9. 9.
      ​data contract experimentation - swirlai

Implementation

  1. 1.
    ​JSON Schema is a vocabulary that allows you to annotate and validate JSON documents, example.
  2. 2.
    protobuf & gRPC
    1. 1.
      ​Protobuf on git, google dev​
    2. 2.
      ​what is gRPC?​
    3. 3.
      ​what are proto buffers, what do they solve and what are the benefits?​
    4. 4.
      gRPC - a basic pythonic tutorial​
    5. 5.
      Protobuf - a pythonic tutorial​
    6. 6.
      (good) Intro to gRPC & Protobuf by Trevor Kendrick​
    7. 7.
      (good) What are Protocol Buffers and why they are widely used? by Dineshchandgr​
    8. 8.
      ​Understanding protobufs by Daniel Padua Ferreira "Protocol Buffers (protobuf) is a method of serializing structured data which is particulary useful to communication between services or storing data. It was designed by Google early 2001 (but only publicly released in 2008) to be smaller and faster than XML. Protobuf messages are serialized into a binary wire format which is very compact and boosts performance."
    9. 9.
      protobuf what and why? by Swaminathan Muthuveerappan​
    10. 10.
      off topic - how to choose between grpc, graphql, rest​
  3. 3.
    managing proto files and other schema types such as avro or json schema, can be done in kafka's schema registry.
DataOps - Previous
Data Mesh
Next - DataOps
Data Product
Last modified 7mo ago