This project is read-only.

NDatabase File Format


NDatabase stores all its data in a single file:
  • Meta-model : classes that are stored in the object base.
  • Objects thats are stored.
  • Indexes.

File format big picture


ODBFileFormat.png

NDatabase file format has the following block types:
  • Indexes are stored in the database as normal objects using the ClassInfoIndex class. This class is marked as a system class (DefaultClassIntrospector.GetClassCategory(String fullClassName)).

Internal storage


In the current version of NDatabase, entities are stored in the database file as two ways linked list.

Let's take an example to understand that:

An object which is managed by a NonNativeObjectInfo (Layer 2) object contains the data of the objects, the OID of the its class (ClassInfo) and some pointers
  • pointer to the next object of the same type,
  • pointer to the previous object of the same type,

This pointers allow the navigation from an object to other. Like a two ways Linked list. This is simple way to handle object navigation but have some drawbacks for the Isolation property of ACID): this issue will be fixed in some future version.

Database Header Block


No Field Description Position Size
1 Version The NDatabase File format version 0 1 int
2 Database ID A unique identifier of the NDatabase database file 4 4 Long
3 Last Transaction sequential ID The last Transaction Sequential ID 36 2 Long
4 Number of classes The number of classes in meta model 36+2*8=52 1 long
5 First ClassInfo OID The OID of the first ClassInfo of the meta model 60 1 long
6 Last ODB Close status To check last close status : 1 is ok, 0 is not 68 1 boolean
7 Empty Space Space which could be used in the future version of NDatabase 69 120 bytes
8 Database String Encoding The Database String Encoding, default is UTF-8 189 50 bytes + 8 bytes for string size = 58 bytes
9 Current block id position The current id block to be used 247 1 Long
10 First ID Block The first block of ids 255 1000 bytes


Header size = 255
Total header size, including first id block of 1000 entries = 255 + 18034 = 18289

ID Block


The ID Block is used to store all ids of objects and classes. In the ID Block, the following data are stored:
  • The Object/class ID
  • The OID type (Object or class)
  • The physical position of the object with the OID
  • An indicator to say if OID is still active. For example deleted objects are marks as deleted

The ODB header holds the position of the first ID block(field 10).

The ID Block can contains up to 1000 OIDs definitions. It has a fixed size :
  • ID Block size = size of header + 1000 * size of a repetition = 34 + 1000 * 18 = 18034 bytes

The ID Blocks are managed by the IdManager class. The actual writing into database file is done by the ObjectWriter .

ID Block Header

No Field Description Position Size
1 Block size The Block size 0 1 Int
2 Block type 20 (NDatabase.Odb.Impl.Core.Layers.Layer3.Block.BlockTypes.BlockTypeIds) 4 1 byte
3 Block status To indicate if block is full,1=no,2=full 5 1 byte
4 Prev block position The previous ID block position 6 1 Long
5 Next block position The next ID block position 14 1 Long
6 Block number The block number 22 1 Int
7 Max id Max id already allocated in this block 26 1 Long

ID Block Repetition

No Field Description Size
1 ID type To indicate if it is an ID of Object or Class 1 byte
2 OID The Object ID 1 Long
3 Status The status of the OID. 0=Unknown, 1=Active, 2=Deleted. See NDatabase.Odb.Impl.Core.Layers.Layer3.Oid.IDStatus 1 byte
4 Position The phisical position of the object with this OID 1 Long

Class Info Block


The meta model of the database is stored using ClassInfo which is a meta-representation of a native class.

Class Info Header

No Field Description Position Size
1 Block size The Block size 0 1 Int
2 Block type 1 (NDatabase.Odb.Impl.Core.Layers.Layer3.Block.BlockTypes.BlockTypeClassHeader) 4 1 byte
3 Class Type System class:1, User class:2 5 1 byte
4 Class id The id of the class 6 1 Long
5 Prev class info OID The OID of the previous Class Info 14 1 Long
6 Next class info OID The OID of the next Class Info 22 1 Long
7 Number of objects The number of object of this class 30 1 Long
8 First object OID The OID of the first object of this class 38 1 Long
9 Last Object OID The OID of the last object of this class 46 1 Long
10 Full class name (with package/namespace) The name of the class 54 Variable
11 Class attribute definition position The position where to find the class attributes definition - 1 Long

Class Info Attributes

When the attribute is a Native Object

No Field Description Position Size
1 Is Native 1 if native 0 1 byte
2 ODB type id NDatabase.Odb.Core.Layers.Layer2.Meta.ODBType 1 1 int
3 Name Attribute name 5 variable
4 Is used in index If attribute is used in an index - 1 byte


When the attribute is a Native Object: array

No Field Description Position Size
1 Is Native 1 if native 0 1 byte
2 ODB type id NDatabase.Odb.Core.Layers.Layer2.Meta.ODBType 1 1 int
3 Sub Type ID The ODB type of the array elements 5 1 int
4 Class Info OID The class info OID if sub type is a NonNative Object. If not, this field does not exist 9 1 int
5 Name Attribute name 9 / 13 variable
6 Is used in index If attribute is used in an index - 1 byte


When the attribute is a Non Native Object: array

No Field Description Position Size
1 Is Native Itis not native : 0 0 1 byte
2 Class Info OID The class info OID of the attribute 1 1 Long
3 Name Attribute name 9 variable
4 Is used in index If attribute is used in an index - 1 byte

Object Block


No Field Description Position Size
1 Block size the size of the block 0 1 int
2 Block type 4 (NDatabase.Odb.Impl.Core.Layers.Layer3.Block.BlockTypes.BlockTypeNonNativeObject) 1 1 byte
3 OID The Object ID 5 1 Long
4 ClassInfo OID The OID of the class of the object 13 1 Long
5 Previous Object OID OID of the previous object 21 1 Long
6 Next Object OID OID if the next Object 29 1 Long
7 Creation Date The Object creation date 37 1 Long
8 Update Date The Object update date 45 1 Long
9 Version number Object version number 53 1 Int
10 Object reference Not used yet. If this objects Is being referenced by others,then this field will be greater>0(v2.0), in version 2.x, it will point to an internal object of type ObjectReference that will have details on the references. This is to enable object integrity.: to enable object integrity 57 1 Long
11 External synchronization indicator Not used yet. True if this object have been synchronized with main database, else false 65 1 boolean
12 Number of attributes Number of attributes of the object 66 1 Int


Then write the attribute data.

First write a block of NbAttributes elements. Each element contains the id of the attribute (int) and the OID or position of the value of the attribute.

No Field Description Position Size
1 Attribute ID this id is the ID of the attribute that was defined in the ClassInfo 0 1 int
2 Attribute value OID How/where to find the value of the attribute. For native objects it is the physical position, for non native objects it is the OID (In this case, the OID, long value, is stored as a negative number to differ from position) 4 1 Long


Then writes the values of the attributes. Values of Native Objects are written in place. Non Native Objects are written at the end of the file.

Lists, arrays and Maps are written in the object block as native objects :
  • Lists

No Field Description Position Size
1 Block type The block type - 1 int
2 Block size The block size - 1 byte
3 Odb Type ID The odb type id (List) - 1 int
4 Is Null? To indicate if object is null - 1 Boolean
5 Class name The real Collection Class Name - String
6 Size Size of the list - Long
7 Block of element identification A block of OID of the elements of the list. If an element is native (ex: a string), the its position is stored instead of OID - block of long

  • Native Attributes

No Field Description Position Size
1 Block Size The block size, fix = 9 - 1 int
2 Block Type The block type - 1 byte
3 Attribute type The odb type id - 1 int
4 Is Null? To indicate if object is null - 1 Boolean
5 The attribute value The bytes of the attribute value - -

String


Strings have a specific way to be written to file. A String in NDatabase have 2 size attributes:
  1. the max size that have been reserved for the string (to enable storing a bigger string in the same place)
  2. the real string size

The bytes that represent the string are encoded using the encoding defined by the OdbConfiguration.GetDatabaseCharacterEncoding(). Default is UTF-8.

No Field Description Position Size
1 Total size Total String size - 1 int
2 Real size Real String size - 1 int
3 The String The bytes of the string - variable


The encoding & decoding are implemented in NDatabase.Odb.Impl.Core.Layers.Layer3.Engine.ByteArrayConverter class.

StorageEngineConstant static class contains some constant value for each field position.

Last edited Aug 18, 2012 at 1:55 PM by jacek, version 27

Comments

No comments yet.