Accessing+huge+files

//by Richard Russell, October 2007//

Most of the //BBC BASIC for Windows// file-handling statements and functions are fully compatible with files in excess of 2 Gbytes (2^31 bytes) in length. You can successfully open (using **OPENIN** or **OPENUP**) files of any length, and files created with **OPENOUT** can exceed 2 Gbytes in length if you write enough data to them! Sequential reads (using **INPUT#**, **BGET#** and **GET$#**) and writes (using **PRINT#** or **BPUT#**) will work correctly whatever the file size, even when crossing the 2 Gbytes 'boundary'.

The only exceptions are **EOF#**, **PTR#** and **EXT#**. When setting the file pointer using **PTR#file=** you can only set it within the first 2 Gbytes of the file, and when setting a file's length using **EXT#file=** you can only set it to a maximum of 2^31-1 bytes. Using **PTR#** to read the current file pointer will work correctly only if the pointer is within the first 2 Gbytes, and you cannot read the file's size using **EXT#** if it exceeds 2 Gbytes. The **EOF#** function will not work reliably if the file length exceeds 2 Gbytes.

Should you need to position the file pointer beyond the first 2 Gbytes, or to read the length of a file bigger than 2 Gbytes, you can use the **PROCptr** and **FNext** routines listed below. Note that the pointer value passed to **PROCptr** and the file size returned from **FNext** necessarily exceed the range of an integer variable, so you should use variant variables to hold these values. If you need the values to be precise (which you probably will, at least in the case of **PROCptr**) you must run your program in ***FLOAT 64** mode.

Here is the replacement for **EXT#**:

code format="bb4w" DEF FNext(F%) LOCAL H%, L%     SYS "GetFileSize", @hfile%(F%), ^H% TO L%      = H% * 2^32 + FNuint(L%) code Note that this function may //not// return the correct value while you are writing to the file; it should normally be used only when the file has been opened for reading.

Here is the replacement for **PTR#**:

code format="bb4w" DEF PROCptr(F%, p)     LOCAL H%, L%      PTR#F% = 0 H% = p / 2^32 L% = FNintu(p - H%*2^32) SYS "SetFilePointer", @hfile%(F%), L%, ^H%, 0 ENDPROC code It is not straightforward to //read// the file pointer (beyond 2 Gbytes) or to //write// the file's length (in excess of 2 Gbytes) so routines to do that are not provided. It will normally be possible to avoid these operations.

The **FNuint** and **FNintu** functions are listed in the article Handling unsigned 32-bit values but are reproduced below for convenience:

code format="bb4w" DEF FNuint(N%) = (N% >>> 1)*2 + (N% AND 1)

DEF FNintu(N) = ((N / 2) << 1) - (INT(N / 2) <> N / 2) code