Delphi Programming Guide
Delphi Programmer 

Menu  Table of contents
Bookmark and Share

Part I - Foundations
  Chapter 1 Delphi 7 and Its IDE
  Chapter 2 The Delphi Programming Language
  Chapter 3 The Run-Time Library
  Chapter 4 Core Library classes
  Chapter 5 Visual Controls
  Chapter 6 Building the User Interface
  Chapter 7 Working with Forms
Part II - Delphi Object-Oriented Architectures
  Chapter 8 The Architecture of Delphi Applications
  Chapter 9 Writing Delphi Components
  Chapter 10 Libraries and Packages
  Chapter 11 Modeling and OOP Programming (with ModelMaker)
  Chapter 12 From COM to COM+
Part III - Delphi Database-Oriented Architectures
  Chapter 13 Delphi's Database Architecture
  Chapter 14 Client/Server with dbExpress
  Chapter 15 Working with ADO
  Chapter 16 Multitier DataSnap Applications
  Chapter 17 Writing Database Components
  Chapter 18 Reporting with Rave
Part IV - Delphi, the Internet, and a .NET Preview
  Chapter 19 Internet Programming: Sockets and Indy
  Chapter 20 Web Programming with WebBroker and WebSnap
  Chapter 21 Web Programming with IntraWeb
  Chapter 22 Using XML Technologies
  Chapter 23 Web Services and SOAP
  Chapter 24 The Microsoft .NET Architecture from the Delphi Perspective
  Chapter 25 Delphi for .NET Preview: The Language and the RTL
  Appendix A Extra Delphi Tools by the Author
  Appendix B Extra Delphi Tools from Other Sources
  Appendix C Free Companion Books on Delphi
  List of Figures    
  List of tables    
  List of Listings    
  List of Sidebars  

Previous Section Next Section

Building Custom Datasets

When discussing the TDataSet class and the alternative families of dataset components available in Delphi in Chapter 13, "Delphi's Database Architecture," I mentioned the possibility of writing a custom dataset class. Now it's time to look at an example. The reasons for writing a custom dataset relate to the fact that you won't need to deploy a database engine, but you'll still be able to take full advantage of Delphi's database architecture, including things like persistent database fields and data-aware controls.

Writing a custom dataset is one of the most complex tasks for a component developer, so this is one of the most advanced areas (as far as low-level coding practices, including tons of pointers) in the book. Moreover, Borland hasn't released any official documentation about writing custom datasets. If you are early in your experience with Delphi, you might want to skip the rest of this chapter and come back later.

TDataSet is an abstract class that declares several virtual abstract methods—23 in Delphi 5, now only a handful, as most have been replaced by empty virtual methods (which you still have to override). Every subclass of TDataSet must override all those methods.

Before discussing the development of a custom dataset, we need to explore a few technical elements of the TDataSet class—in particular, record buffering. The class maintains a list of buffers that store the values of different records. These buffers store the data, but they also usually store further information for the dataset to use when managing the records. These buffers don't have a predefined structure, and each custom dataset must allocate the buffers, fill them, and destroy them. The custom dataset must also copy the data from the record buffers to the various fields of the dataset, and vice versa. In other words, the custom dataset is entirely responsible for handling these buffers.

In addition to managing the data buffers, the component is responsible for navigating among the records, managing the bookmarks, defining the structure of the dataset, and creating the proper data fields. The TDataSet class is nothing more than a framework; you must fill it with the appropriate code. Fortunately, most of the code follows a standard structure, which the TDataSet-derived VCL classes use. Once you've grasped the key ideas, you'll be able to build multiple custom datasets borrowing quite a lot of code.

To simplify this type of reuse, I've collected the common features required by any custom dataset in a TMdCustomDataSet class. However, I'm not going to discuss the base class first and the specific implementation later, because it would be difficult to understand. Instead, I'll detail the code required by a dataset, presenting methods of the generic and specific classes at the same time, according to a logical flow.

The Definition of the Classes

The starting point, as usual, is the declaration of the two classes discussed in this section: the generic custom dataset I've written and a specific component storing data in a file stream. The declaration of these classes is available in Listing 17.2. In addition to virtual methods, the classes contain a series of protected fields used to manage the buffers, track the current position and record count, and handle many other features. You'll also notice another record declaration at the beginning: a structure used to store the extra data for every data record you place in a buffer. The dataset places this information in each record buffer, following the data.

Listing 17.2: The Declaration of TMdCustomDataSet and TMdDataSetStream
Start example
// in the unit MdDsCustom
  EMdDataSetError = class (Exception);
  TMdRecInfo = record
    Bookmark: Longint;
    BookmarkFlag: TBookmarkFlag;
  PMdRecInfo = ^TMdRecInfo;
  TMdCustomDataSet = class(TDataSet)
    // status
    FIsTableOpen: Boolean;
    // record data
    FRecordSize,        // the size of the actual data
    FRecordBufferSize,  // data + housekeeping (TRecInfo)
    FCurrentRecord,     // current record (0 to FRecordCount - 1)
    BofCrack,           // before the first record (crack)
    EofCrack: Integer;  // after the last record (crack)
    // create, close, and so on
    procedure InternalOpen; override;
    procedure InternalClose; override;
    function IsCursorOpen: Boolean; override;
    // custom functions
    function InternalRecordCount: Integer; virtual; abstract;
    procedure InternalPreOpen; virtual;
    procedure InternalAfterOpen; virtual;
    procedure InternalLoadCurrentRecord(Buffer: PChar); virtual; abstract;
    // memory management
    function AllocRecordBuffer: PChar; override;
    procedure InternalInitRecord(Buffer: PChar); override;
    procedure FreeRecordBuffer(var Buffer: PChar); override;
    function GetRecordSize: Word; override;
    // movement and optional navigation (used by grids)
    function GetRecord(Buffer: PChar; GetMode: TGetMode; DoCheck: Boolean):
      TGetResult; override;
    procedure InternalFirst; override;
    procedure InternalLast; override;
    function GetRecNo: Longint; override;
    function GetRecordCount: Longint; override;
    procedure SetRecNo(Value: Integer); override;
    // bookmarks
    procedure InternalGotoBookmark(Bookmark: Pointer); override;
    procedure InternalSetToRecord(Buffer: PChar); override;
    procedure SetBookmarkData(Buffer: PChar; Data: Pointer); override;
    procedure GetBookmarkData(Buffer: PChar; Data: Pointer); override;
    procedure SetBookmarkFlag(Buffer: PChar; Value: TBookmarkFlag); override;
    function GetBookmarkFlag(Buffer: PChar): TBookmarkFlag; override;
    // editing (dummy versions)
    procedure InternalDelete; override;
    procedure InternalAddRecord(Buffer: Pointer; Append: Boolean); override;
    procedure InternalPost; override;
    procedure InternalInsert; override;
    // other
    procedure InternalHandleException; override;
    // redeclared dataset properties
    property Active;
    property BeforeOpen;
    property AfterOpen;
    property BeforeClose;
    property AfterClose;
    property BeforeInsert;
    property AfterInsert;
    property BeforeEdit;
    property AfterEdit;
    property BeforePost;
    property AfterPost;
    property BeforeCancel;
    property AfterCancel;
    property BeforeDelete;
    property AfterDelete;
    property BeforeScroll;
    property AfterScroll;
    property OnCalcFields;
    property OnDeleteError;
    property OnEditError;
    property OnFilterRecord;
    property OnNewRecord;
    property OnPostError;
// in the unit MdDsStream
  TMdDataFileHeader = record
    VersionNumber: Integer;
    RecordSize: Integer;
    RecordCount: Integer;
  TMdDataSetStream = class(TMdCustomDataSet)
    procedure SetTableName(const Value: string);
    FDataFileHeader: TMdDataFileHeader;
    FDataFileHeaderSize,    // optional file header size
    FRecordCount: Integer;  // current number of records
    FStream: TStream;       // the physical table
    FTableName: string;     // table path and file name
    FFieldOffset: TList;    // field offsets in the buffer
    // open and close
    procedure InternalPreOpen; override;
    procedure InternalAfterOpen; override;
    procedure InternalClose; override;
    procedure InternalInitFieldDefs; override;
    // edit support
    procedure InternalAddRecord(Buffer: Pointer; Append: Boolean); override;
    procedure InternalPost; override;
    procedure InternalInsert; override;
    // fields
    procedure SetFieldData(Field: TField; Buffer: Pointer); override;
    // custom dataset virutal methods
    function InternalRecordCount: Integer; override;
    procedure InternalLoadCurrentRecord(Buffer: PChar); override;
    procedure CreateTable;
    function GetFieldData(Field: TField; Buffer: Pointer): Boolean; override;
    property TableName: string read FTableName write SetTableName;
End example

When I divided the methods into sections (as you can see by looking at the source code files), I marked each one with a roman number. You'll see those numbers in a comment describing the method, so that while browsing this long listing you'll immediately know which of the four sections you are in.

Section I: Initialization, Opening, and Closing

The first methods I'll examine are responsible for initializing the dataset and for opening and closing the file stream used to store the data. In addition to initializing the component's internal data, these methods are responsible for initializing and connecting the proper TFields objects to the dataset component. To make this work, all you need to do is to initialize the FieldsDef property with the definitions of the fields for your dataset, and then call a few standard methods to generate and bind the TField objects. This is the general InternalOpen method:

procedure TMdCustomDataSet.InternalOpen;
  InternalPreOpen; // custom method for subclasses
  // initialize the field definitions
  // if there are no persistent field objects, create the fields dynamically
  if DefaultFields then
  // connect the TField objects with the actual fields
  BindFields (True);
  InternalAfterOpen; // custom method for subclasses
  // sets cracks and record position and size
  BofCrack := -1;
  EofCrack := InternalRecordCount;
  FCurrentRecord := BofCrack;
  FRecordBufferSize := FRecordSize + sizeof (TMdRecInfo);
  BookmarkSize := sizeOf (Integer);
  // everything OK: table is now open
  FIsTableOpen := True;

You'll notice that the method sets most of the local fields of the class, and also the BookmarkSize field of the base TDataSet class. Within this method, I call two custom methods I introduced in my custom dataset hierarchy: InternalPreOpen and InternalAfterOpen. The first, InternalPreOpen, is used for operations required at the very beginning, such as checking whether the dataset can be opened and reading the header information from the file. The code checks an internal version number for consistency with the value saved when the table is first created, as you'll see later. By raising an exception in this method, you can eventually stop the open operation.

Here is the code for the two methods in the derived stream-based dataset:

  HeaderVersion = 10;
procedure TMdDataSetStream.InternalPreOpen;
  // the size of the header
  FDataFileHeaderSize := sizeOf (TMdDataFileHeader);
  // check if the file exists
  if not FileExists (FTableName) then
    raise EMdDataSetError.Create ('Open: Table file not found');
  // create a stream for the file
  FStream := TFileStream.Create (FTableName, fmOpenReadWrite);
  // initialize local data (loading the header)
  FStream.ReadBuffer (FDataFileHeader, FDataFileHeaderSize);
  if FDataFileHeader.VersionNumber <> HeaderVersion then
    raise EMdDataSetError.Create ('Illegal File Version');
  // let's read this, double check later
  FRecordCount := FDataFileHeader.RecordCount;
procedure TMdDataSetStream.InternalAfterOpen;
  // check the record size
  if FDataFileHeader.RecordSize <> FRecordSize then
    raise EMdDataSetError.Create ('File record size mismatch');
  // check the number of records against the file size
 if (FDataFileHeaderSize + FRecordCount * FRecordSize) <> FStream.Size then
    raise EMdDataSetError.Create ('InternalOpen: Invalid Record Size');

The second method, InternalAfterOpen, is used for operations required after the field definitions have been set and is followed by code that compares the record size read from the file against the value computed in the InternalInitFieldDefs method. The code also checks that the number of records read from the header is compatible with the size of the file. This test can fail if the dataset wasn't closed properly: You might want to modify this code to let the dataset refresh the record size in the header anyway.

The InternalOpen method of the custom dataset class is specifically responsible for calling InternalInitFieldDefs, which determines the field definitions (at either design time or run time). For this example, I decided to base the field definitions on an external file—an INI file that provides a section for every field. Each section contains the name and data type of the field, as well as its size if it is string data. Listing 17.3 shows the Contrib.INI file used in the component's demo application.

Listing 17.3: The Contrib.INI File for the Demo Application
Start example
Number = 6
Type = ftString
Name = Name
Size = 30
Type = ftInteger
Name = Level
Type = ftDate
Name = BirthDate
Type = ftCurrency
Name = Stipend
Type = ftString
Name = Email
Size = 50
Type = ftBoolean
Name = Editor
End example

This file, or a similar one, must use the same name as the table file and must be in the same directory. The InternalInitFieldDefs method (shown in Listing 17.4) will read it, using the values it finds to set up the field definitions and determine the size of each record. The method also initializes an internal TList object that stores the offset of every field inside the record. You use this TList to access fields' data within the record buffer, as you can see in the code listing.

Listing 17.4: The InternalInitFieldDefs Method of the Stream-Based Dataset
Start example
procedure TMdDataSetStream.InternalInitFieldDefs;
  IniFileName, FieldName: string;
  IniFile: TIniFile;
  nFields, I, TmpFieldOffset, nSize: Integer;
  FieldType: TFieldType;
  FFieldOffset := TList.Create;
  TmpFieldOffset := 0;
  IniFilename := ChangeFileExt(FTableName, '.ini');
  Inifile := TIniFile.Create (IniFilename);
  // protect INI file
    nFields := IniFile.ReadInteger (' Fields', 'Number', 0);
    if nFields = 0 then
      raise EDataSetOneError.Create (' InitFieldsDefs: 0 fields?');
    for I := 1 to nFields do
      // create the field
      FieldType := TFieldType (GetEnumValue (TypeInfo (TFieldType),
        IniFile.ReadString ('Field' + IntToStr (I), 'Type', '')));
      FieldName := IniFile.ReadString ('Field' + IntToStr (I), 'Name', '');
      if FieldName = ''  then
        raise EDataSetOneError.Create (
          'InitFieldsDefs: No name for field ' + IntToStr (I));
      nSize := IniFile.ReadInteger ('Field' + IntToStr (I), 'Size', 0);
      FieldDefs.Add (FieldName, FieldType, nSize, False);
      // save offset and compute size
      FFieldOffset.Add (Pointer (TmpFieldOffset));
      case FieldType of
        ftString:                         Inc (TmpFieldOffset, nSize + 1);
        ftBoolean, ftSmallInt, ftWord:    Inc (TmpFieldOffset, 2);
        ftInteger, ftDate, ftTime:        Inc (TmpFieldOffset, 4);
        ftFloat, ftCurrency, ftDateTime:  Inc (TmpFieldOffset, 8);
        raise EDataSetOneError.Create (
           'InitFieldsDefs: Unsupported field type');
    end; // for
  FRecordSize := TmpFieldOffset;
End example

Closing the table is a matter of disconnecting the fields (using some standard calls). Each class must dispose of the data it allocated and update the file header, the first time records are added and each time the record count has changed:

procedure TMdCustomDataSet.InternalClose;
  // disconnect field objects
  BindFields (False);
  // destroy field object (if not persistent)
  if DefaultFields then
  // close the file
  FIsTableOpen := False;
procedure TMdDataSetStream.InternalClose;
  // if required, save updated header
  if (FDataFileHeader.RecordCount <> FRecordCount) or
    (FDataFileHeader.RecordSize = 0) then
    FDataFileHeader.RecordSize := FRecordSize;
    FDataFileHeader.RecordCount := FRecordCount;
    if Assigned (FStream) then
      FStream.Seek (0, soFromBeginning);
      FStream.WriteBuffer (FDataFileHeader, FDataFileHeaderSize);
  // free the internal list field offsets and the stream
  inherited InternalClose;

Another related function is used to test whether the dataset is open, something you can solve using the corresponding local field:

function TMdCustomDataSet.IsCursorOpen: Boolean;
  Result := FIsTableOpen;

These are the opening and closing methods you need to implement in any custom dataset. However, most of the time, you'll also add a method to create the table. In this example, the CreateTable method creates an empty file and inserts information in the header: a fixed version number, a dummy record size (you don't know the size until you initialize the fields), and the record count (which is zero to start):

procedure TMdDataSetStream.CreateTable;
  // create the new file
  if FileExists (FTableName) then
    raise EMdDataSetError.Create ('File ' + FTableName + ' already exists');
  FStream := TFileStream.Create (FTableName, fmCreate or fmShareExclusive);
    // save the header
    FDataFileHeader.VersionNumber := HeaderVersion;
    FDataFileHeader.RecordSize := 0;    // used later
    FDataFileHeader.RecordCount := 0;   // empty
    FStream.WriteBuffer (FDataFileHeader, FDataFileHeaderSize);
    // close the file

Section II: Movement and Bookmark Management

As mentioned earlier, every dataset must implement bookmark management, which is necessary for navigating through the dataset. Logically, a bookmark is a reference to a specific dataset record, something that uniquely identifies the record so a dataset can access it and compare it to other records. Technically, bookmarks are pointers. You can implement them as pointers to specific data structures that store record information, or you can implement them as record numbers. For simplicity, I'll use the latter approach.

Given a bookmark, you should be able to find the corresponding record; but given a record buffer, you should also be able to retrieve the corresponding bookmark. This is the reason for appending the TMdRecInfo structure to the record data in each record buffer. This data structure stores the bookmark for the record in the buffer, as well as some bookmark flags defined as follows:

  TBookmarkFlag = (bfCurrent, bfBOF, bfEOF, bfInserted);

The system will request that you store these flags in each record buffer and will later ask you to retrieve the flags for a given record buffer.

To summarize, the structure of a record buffer stores the record data, the bookmark, and the bookmark flags, as you can see in Figure 17.5.

Click To expand
Figure 17.5:  The structure of each buffer of the custom dataset, along with the various local fields referring to its subportions

To access the bookmark and flags, you can use as an offset the size of the data, casting the value to the PMdRecInfo pointer type, and then access the proper field of the TMdRecInfo structure via the pointer. The two methods used to set and get the bookmark flags demonstrate this technique:

procedure TMdCustomDataSet.SetBookmarkFlag (Buffer: PChar;
  Value: TBookmarkFlag);
  PMdRecInfo(Buffer + FRecordSize).BookmarkFlag := Value;
function TMdCustomDataSet.GetBookmarkFlag (Buffer: PChar): TBookmarkFlag;
  Result := PMdRecInfo(Buffer + FRecordSize).BookmarkFlag;

The methods you use to set and get a record's current bookmark are similar to the previous two, but they add complexity because you receive a pointer to the bookmark in the Data parameter. Casting the value referenced by this pointer to an integer, you obtain the bookmark value:

procedure TMdCustomDataSet.GetBookmarkData (Buffer: PChar; Data: Pointer);
  Integer(Data^) := PMdRecInfo(Buffer + FRecordSize).Bookmark;
procedure TMdCustomDataSet.SetBookmarkData (Buffer: PChar; Data: Pointer);
  PMdRecInfo(Buffer + FRecordSize).Bookmark := Integer(Data^);

The key bookmark management method is InternalGotoBookmark, which your dataset uses to make a given record the current one. This isn't the standard navigation technique—it's much more common to move to the next or previous record (something you can accomplish using the GetRecord method presented in the next section), or to move to the first or last record (something you'll accomplish using the InternalFirst and InternalLast methods described shortly).

Oddly enough, the InternalGotoBookmark method doesn't expect a bookmark parameter, but a pointer to a bookmark, so you must dereference it to determine the bookmark value. You use the following method, InternalSetToRecord, to jump to a given bookmark, but it must extract the bookmark from the record buffer passed as a parameter. Then, InternalSetToRecord calls InternalGotoBookmark. Here are the two methods:

procedure TMdCustomDataSet.InternalGotoBookmark (Bookmark: Pointer);
  ReqBookmark: Integer;
  ReqBookmark := Integer (Bookmark^);
  if (ReqBookmark >= 0) and (ReqBookmark < InternalRecordCount) then
    FCurrentRecord := ReqBookmark
    raise EMdDataSetError.Create ('Bookmark ' +
      IntToStr (ReqBookmark) + ' not found');
procedure TMdCustomDataSet.InternalSetToRecord (Buffer: PChar);
  ReqBookmark: Integer;
  ReqBookmark := PMdRecInfo(Buffer + FRecordSize).Bookmark;
  InternalGotoBookmark (@ReqBookmark);

In addition to the bookmark management methods just described, you use several other navigation methods to move to specific positions within the dataset, such as the first or last record. These two methods don't really move the current record pointer to the first or last record, but move it to one of two special locations before the first record and after the last one. These are not actual records: Borland calls them cracks. The beginning-of-file crack, or BofCrack, has the value –1 (set in the InternalOpen method), because the position of the first record is zero. The end-of-file crack, or EofCrack, has the value of the number of records, because the last record has the position FRecordCount - 1. I used two local fields, called EofCrack and BofCrack, to make this code easier to read:

procedure TMdCustomDataSet.InternalFirst;
  FCurrentRecord := BofCrack;
procedure TMdCustomDataSet.InternalLast;
  EofCrack := InternalRecordCount;
  FCurrentRecord := EofCrack;

The InternalRecordCount method is a virtual method introduced in my TMdCustomDataSet class, because different datasets can either have a local field for this value (as in case of the stream-based dataset, which has an FRecordCount field) or compute it on the fly.

Another group of optional methods is used to get the current record number (used by the DBGrid component to show a proportional vertical scroll bar), set the current record number, or determine the number of records. These methods are easy to understand, if you recall that the range of the internal FCurrentRecord field is from 0 to the number of records minus 1. In contrast, the record number reported to the system ranges from 1 to the number of records:

function TMdCustomDataSet.GetRecordCount: Longint;
  Result := InternalRecordCount;
function TMdCustomDataSet.GetRecNo: Longint;
  if FCurrentRecord < 0 then
    Result := 1
    Result := FCurrentRecord + 1;
procedure TMdCustomDataSet.SetRecNo(Value: Integer);
  if (Value > 1) and (Value <= FRecordCount) then
    FCurrentRecord := Value - 1;

Notice that the generic custom dataset class implements all the methods of this section. The derived stream-based dataset doesn't need to modify any of them.

Section III: Record Buffers and Field Management

Now that we've covered all the support methods, let's examine the core of a custom dataset. In addition to opening and creating records and moving around between them, the component needs to move the data from the stream (the persistent file) to the record buffers, and from the record buffers to the TField objects that are connected to the data-aware controls. The management of record buffers is complex, because each dataset also needs to allocate, empty, and free the memory it requires:

function TMdCustomDataSet.AllocRecordBuffer: PChar;
  GetMem (Result, FRecordBufferSize);
procedure TMdCustomDataSet.FreeRecordBuffer (var Buffer: PChar);
  FreeMem (Buffer);

You allocate memory this way because a dataset generally adds more information to the record buffer, so the system has no way of knowing how much memory to allocate. Notice that in the AllocRecordBuffer method, the component allocates the memory for the record buffer, including both the database data and the record information. In the InternalOpen method, I wrote the following:

FRecordBufferSize := InternalRecordSize + sizeof (TMdRecInfo);

The component also needs to implement a function to reset the buffer (InternalInitRecord), usually filling it with numeric zeros or spaces.

Oddly enough, you must also implement a method that returns the size of each record, but only the data portion—not the entire record buffer. This method is necessary for implementing the read-only RecordSize property, which is used only in a couple of peculiar cases in the entire VCL source code. In the generic custom dataset, the GetRecordSize method returns the value of the FRecordSize field.

Now we've reached the core of the custom dataset component. The methods in this group are GetRecord, which reads data from the file; InternalPost and InternalAddRecord, which update or add new data to the file; and InternalDelete, which removes data and is not implemented in the sample dataset.

The most complex method of this group is GetRecord, which serves multiple purposes. The system uses this method to retrieve the data for the current record, fill a buffer passed as a parameter, and retrieve the data of the next or previous records. The GetMode parameter determines its action:

  TGetMode = (gmCurrent, gmNext, gmPrior);

Of course, a previous or next record might not exist. Even the current record might not exist—for example, when the table is empty (or in case of an internal error). In these cases you don't retrieve the data but return an error code. Therefore, this method's result can be one of the following values:

  TGetResult = (grOK, grBOF, grEOF, grError);

Checking to see if the requested record exists is slightly different than you might expect. You don't have to determine if the current record is in the proper range, only if the requested record is. For example, in the gmCurrent branch of the case statement, you use the standard expression CurrentRecord>= InternalRecourdCount. To fully understand the various cases, you might want to read the code a couple of times.

It took me some trial and error (and system crashes caused by recursive calls) to get the code straight when I wrote my first custom dataset a few years back. To test it, consider that if you use a DBGrid, the system will perform a series of GetRecord calls, until either the grid is full or GetRecord return grEOF. Here's the entire code for the GetRecord method:

// III: Retrieve data for current, previous, or next record
// (moving to it if necessary) and return the status
function TMdCustomDataSet.GetRecord(Buffer: PChar;
  GetMode: TGetMode; DoCheck: Boolean): TGetResult;
  Result := grOK; // default
  case GetMode of
    gmNext: // move on
      if FCurrentRecord < InternalRecordCount - 1 then
        Inc (FCurrentRecord)
        Result := grEOF; // end of file
    gmPrior: // move back
      if FCurrentRecord > 0 then
        Dec (FCurrentRecord)
        Result := grBOF; // begin of file
    gmCurrent: // check if empty
      if FCurrentRecord >= InternalRecordCount then
        Result := grError;
  // load the data
  if Result = grOK then
    InternalLoadCurrentRecord (Buffer)
  else if (Result = grError) and DoCheck then
    raise EMdDataSetError.Create ('GetRecord: Invalid record');

If there's an error and the DoCheck parameter was True, GetRecord raises an exception. If everything goes fine during record selection, the component loads the data from the stream, moving to the position of the current record (given by the record size multiplied by the record number). In addition, you need to initialize the buffer with the proper bookmark flag and bookmark (or record number) value. This is accomplished by another virtual method I introduced, so that derived classes will only need to implement this portion of the code, while the complex GetRecord method remains unchanged:

procedure TMdDataSetStream.InternalLoadCurrentRecord (Buffer: PChar);
  FStream.Position := FDataFileHeaderSize + FRecordSize * FCurrentRecord;
  FStream.ReadBuffer (Buffer^, FRecordSize);
  with PMdRecInfo(Buffer + FRecordSize)^ do
    BookmarkFlag := bfCurrent;
    Bookmark := FCurrentRecord;

You move data to the file in two different cases: when you modify the current record (that is, a post after an edit) or when you add a new record (a post after an insert or append). You use the InternalPost method in both cases, but you can check the dataset's State property to determine which type of post you're performing. In both cases, you don't receive a record buffer as a parameter; so, you must use the ActiveRecord property of TDataSet, which points to the buffer for the current record:

procedure TMdDataSetStream.InternalPost;
  if State = dsEdit then
    // replace data with new data
    FStream.Position := FDataFileHeaderSize + FRecordSize * FCurrentRecord;
    FStream.WriteBuffer (ActiveBuffer^, FRecordSize);
    // always append
    FStream.Seek (0, soFromEnd);
    FStream.WriteBuffer (ActiveBuffer^, FRecordSize);
    Inc (FRecordCount);

In addition, there's another related method: InternalAddRecord. This method is called by the AddRecord method, which in turn is called by InsertRecord and AppendRecord. These last two are public methods a user can call. This is an alternative to inserting or appending a new record to the dataset, editing the values of the various fields, and then posting the data, because the InsertRecord and AppendRecord calls receive the values of the fields as parameters. All you must do at that point is replicate the code used to add a new record in the InternalPost method:

procedure TMdDataSetOne.InternalAddRecord(Buffer: Pointer; Append: Boolean);
  // always append at the end
  FStream.Seek (0, soFromEnd);
  FStream.WriteBuffer (ActiveBuffer^, FRecordSize);
  Inc (FRecordCount);

I should also have implemented a file operation that removes the current record. This operation is common, but it is complex. If you take a simple approach, such as creating an empty spot in the file, then you'll need to keep track of that spot and make the code for reading or writing a specific record work around that location. An alternate solution is to make a copy of the entire file without the given record and then replace the original file with the copy. Given these choices, I felt that for this example I could forgo supporting record deletion.

Section IV: From Buffers to Fields

In the last few methods, you've seen how datasets move data from the data file to the memory buffer. However, there's little Delphi can do with this record buffer, because it doesn't yet know how to interpret the data in the buffer. You need to provide two more methods: GetData, which copies the data from the record buffer to the field objects of the dataset, and SetData, which moves the data back from the fields to the record buffer. Delphi will automatically move the data from the field objects to the data-aware controls and back.

The code for these two methods isn't difficult, primarily because you saved the field offsets inside the record data in a TList object called FFieldOffset. By incrementing the pointer to the initial position in the record buffer of the current field's offset, you can get the specific data, which takes Field.DataSize bytes.

A confusing element of these two methods is that they both accept a Field parameter and a Buffer parameter. At first, you might think the buffer passed as a parameter is the record buffer. However, I found out that the Buffer is a pointer to the field object's raw data. If you use one of the field object's methods to move that data, it will call the dataset's GetData or SetData method, probably causing an infinite recursion. Instead, you should use the ActiveBuffer pointer to access the record buffer, use the proper offset to get to the data for the current field in the record buffer, and then use the provided Buffer to access the field data. The only difference between the two methods is the direction you move the data:

function TMdDataSetOne.GetFieldData (Field: TField; Buffer: Pointer): Boolean;
  FieldOffset: Integer;
  Ptr: PChar;
  Result := False;
  if not IsEmpty and (Field.FieldNo > 0) then
    FieldOffset := Integer (FFieldOffset [Field.FieldNo - 1]);
    Ptr := ActiveBuffer;
    Inc (Ptr, FieldOffset);
    if Assigned (Buffer) then
      Move (Ptr^, Buffer^, Field.DataSize);
    Result := True;
    if (Field is TDateTimeField) and (Integer(Ptr^) = 0) then
      Result := False;
procedure TMdDataSetOne.SetFieldData(Field: TField; Buffer: Pointer);
  FieldOffset: Integer;
  Ptr: PChar;
  if Field.FieldNo >= 0 then
    FieldOffset := Integer (FFieldOffset [Field.FieldNo - 1]);
    Ptr := ActiveBuffer;
    Inc (Ptr, FieldOffset);
    if Assigned (Buffer) then
      Move (Buffer^, Ptr^, Field.DataSize)
      raise Exception.Create (
        'Very bad error in TMdDataSetStream.SetField data');
    DataEvent (deFieldChange, Longint(Field));

The GetField method should return True or False to indicate whether the field contains data or is empty (a null field, to be more precise). However, unless you use a special marker for blank fields, it's difficult to determine this condition, because you're storing values of different data types. For example, a test such as Ptr^<>#0 makes sense only if you are using a string representation for all the fields. If you use this test, zero integer values and empty strings will show as null values (the data-aware controls will be empty), which may be what you want. The problem is that Boolean False values won't show up. Even worse, floating-point values with no decimals and few digits won't be displayed, because the exponent portion of their representation will be zero. However, to make this example work, I had to consider as empty each date/time field with an initial zero. Without this code, Delphi tries to convert the illegal internal zero date (internally, date fields don't use a TDateTime data type but a different representation), raising an exception. The code used to work with past versions of Delphi.


While trying to fix this problem, I also found out that if you call IsNull for a field, this request is resolved by calling GetFieldData without passing any buffer to fill but looking only for the result of the function call. This is the reason for the if Assigned (Buffer) test within the code.

There's one final method, which doesn't fall into any category: InternalHandleException. Generally, this method silences the exception, because it is activated only at design time.

Testing the Stream-Based DataSet

After all this work, you're ready to test an application example of the custom dataset component, which is installed in the component's package for this chapter. The form displayed by the StreamDSDemo program is simple, as you can see in Figure 17.6. It has a panel with two buttons, a check box, and a navigator component, plus a DBGrid filling its client area.

Click To expand
Figure 17.6:   The form of the StreamDSDemo example. The custom dataset has been activated, so you can already see the data at design time.

Figure 17.6 shows the example's form at design time, but I activated the custom dataset so that its data is visible. I already prepared the INI file with the table definition (the file listed earlier when discussing the dataset initialization), and I executed the program to add some data to the file.

You can also modify the form using Delphi's Fields editor and set the properties of the various field objects. Everything works as it does with one of the standard dataset controls. However, to make this work, you must enter the name of the custom dataset's file in the TableName property, using the complete path.


The demo program defines the absolute path of the table file at design time, so you'll need to fix it if you copy the examples to a different drive or directory. In the example, the TableName property is used only at design time. At run time, the program looks for the table in the current directory.

The example code is simple, especially compared to the custom dataset code. If the table doesn't exist yet, you can click the Create New Table button:

procedure TForm1.Button1Click(Sender: TObject);
  CheckBox1.Checked := MdDataSetStream1.Active;

You create the file first, opening and closing it within the CreateTable call, and then open the table. This is the same behavior as the TTable component (which accomplishes this step using the CreateTable method). To open or close the table, you can click the check box:

procedure TForm1.CheckBox1Click(Sender: TObject);
  MdDataSetStream1.Active := CheckBox1.Checked;

Finally, I created a method that tests the custom dataset's bookmark management code (it works).

Previous Section Next Section



Copyright © 2004-2016 "Delphi Sources". Delphi Programming Guide
     Twitter     Facebook