Protocol Buffer Basics: Java

1.   With protocol buffers, you write a .proto description of the data structure you wish to store. From that, the protocol buffer compiler creates a class that implements automatic encoding and parsing of the protocol buffer data with an efficient binary format. The generated class provides getters and setters for the fields that make up a protocol buffer and takes care of the details of reading and writing the protocol buffer as a unit. Importantly, the protocol buffer format supports the idea of extending the format over time in such a way that the code can still read data encoded with the old format.

 

2.  T he definitions in a .proto file are simple: you add a message for each data structure you want to serialize, then specify a name and a type for each field in the message. . A message is just an aggregate containing a set of typed fields. Many standard simple data types are available as field types, including bool , int32 , float , double , and string . You can also add further structure to your messages by using other message types as field types

 

3.   A value for a required field must be provided, otherwise the message will be considered "uninitialized". Trying to build an uninitialized message will throw a RuntimeException . Parsing an uninitialized message will throw an IOException .

 

4.   For embedded messages, the default value is always the "default instance" or "prototype" of the message, which has none of its fields set.

 

5.   Each message class has its own Builder class that you use to create instances of that class. Both messages and builders have auto-generated accessor methods for each field of the message; messages have only getters while builders have both getters and setters. There are simple JavaBeans-style getters and setters for each field. There are also has getters for each singular field which return true if that field has been set. Finally, each field has a clear method that un-sets the field back to its empty state.

 

6.   Even though the .proto file uses lowercase-with-underscores, the generated classes match standard Java style conventions. You should always use lowercase-with-underscores for field names in your .proto files; this ensures good naming practice in all the generated languages.

 

7.  T he message classes generated by the protocol buffer compiler are all immutable. Once a message object is constructed, it cannot be modified. To construct a message, you must first construct a builder, set any fields you want to set to your chosen values, then call the builder's build() method.

 

8 .   Each method of the builder which modifies the message returns another builder. The returned object is actually the same builder on which you called the method. It is returned for convenience so that you can string several setters together on a single line of code.

 

9 .   Each message and builder class also contains a number of other methods that let you check or manipulate the entire message, including:

  a)   isInitialized() : checks if all the required fields have been set.

  b)   toString() : returns a human-readable representation of the message.

  c)   mergeFrom(Message other) : (builder only) merges the contents of other into this message, overwriting singular fields and concatenating repeated ones.

  d)   clear() : (builder only) clears all the fields back to the empty state.

These methods implement the Message and Message.Builder interfaces shared by all Java messages and builders.

 

10.   Each protocol buffer class has methods for writing and reading messages of your chosen type using the protocol buffer binary format . These include:

  a)   byte[] toByteArray() : serializes the message and returns a byte array containing its raw bytes.

  b)   static Person parseFrom(byte[] data) : parses a message from the given byte array.

  c)   void writeTo(OutputStream output) : serializes the message and writes it to an OutputStream .

  d)   static Person parseFrom(InputStream input) : reads and parses a message from an InputStream .

 

11.  I f you want your new buffers to be backwards-compatible, and your old buffers to be forward-compatible – and you almost certainly do want this – then there are some rules you need to follow. In the new version of the protocol buffer:

  a)   you must not change the tag numbers of any existing fields.

  b)   you must not add or delete any required fields.

  c)   you may delete optional or repeated fields.

  d)   you may add new optional or repeated fields but you must use fresh tag numbers (i.e. tag numbers that were never used in this protocol buffer, not even by deleted fields).

If you follow these rules, old code will happily read new messages and simply ignore any new fields. To the old code, optional fields that were deleted will simply have their default value, and deleted repeated fields will be empty. New code will also transparently read old messages.

 

12.   One key feature provided by protocol message classes is reflection . You can iterate over the fields of a message and manipulate their values without writing your code against any specific message type. One very useful way to use reflection is for converting protocol messages to and from other encodings, such as XML or JSON. A more advanced use of reflection might be to find differences between two messages of the same type, or to develop a sort of "regular expressions for protocol messages" in which you can write expressions that match certain message contents.