[Top] [Up] [Previous] [Next] [Index]

# 10 Streams

### Sections

1. Categories for Streams and the StreamsFamily
2. Operations applicable to All Streams
3. Operations for Input Streams
4. Operations for Output Streams
5. File Streams
6. User Streams
7. String Streams
8. Input-Output Streams
9. Dummy Streams
10. Handling of Streams in the Background

Streams provide flexible access to GAP's input and output processing. An input stream takes characters from some source and delivers them to GAP which reads them from the stream. When an input stream has delivered all characters it is at `end-of-stream`. An output stream receives characters from GAP which writes them to the stream, and delivers them to some destination.

A major use of streams is to provide efficient and flexible access to files. Files can be read and written using `Read` and `AppendTo`, however the former only allows a complete file to be read as GAP input and the latter imposes a high time penalty if many small pieces of output are written to a large file. Streams allow input files in other formats to be read and processed, and files to be built up efficiently from small pieces of output. Streams may also be used for other purposes, for example to read from and print to GAP strings, or to read input directly from the user.

Any stream is either a text stream, which translates the `end-of-line` character (`'\n'`) to or from the system's representation of `end-of-line` (e.g., new-line under UNIX, carriage-return under MacOS, carriage-return-new-line under DOS), or a binary stream, which does not translate the `end-of-line` character. The processing of other unprintable characters by text streams is undefined. Binary streams pass them unchanged.

Whereas it is cheap to append to a stream, streams do consume system resources, and only a limited number can be open at any time, therefore it is necessary to close a stream as soon as possible using `CloseStream` described in Section CloseStream. If creating a stream failed then `LastSystemError` (see LastSystemError) can be used to get information about the failure.

## 10.1 Categories for Streams and the StreamsFamily

- `IsStream(` obj`) C`

  Streams are GAP objects and all open streams, input, output, text and binary, lie in this category.

- `IsClosedStream(` obj`) C`

  When a stream is closed, its type changes to lie in 'IsClosedStream'. This category is used to install methods that trap accesses to closed streams.

- `IsInputStream(` obj`) C`

  All input streams lie in this category, and support input operations such as `ReadByte` (see Operations for Input Streams)

- `IsInputTextStream(` obj`) C`

  All text input streams lie in this category. They translate new-line characters read.

- `IsInputTextNone(` obj`) C`

  It is convenient to use a category to distinguish dummy streams (see Dummy Streams) from others. Other distinctions are usually made using representations

- `IsOutputStream(` obj`) C`

  All output streams lie in this category and support basic operations such as `WriteByte` (see Operations for Output Streams)

- `IsOutputTextStream(` obj`) C`

  All text output streams lie in this category and translate new-line characters on output.

- `IsOutputTextNone(` obj`) C`

  It is convenient to use a category to distinguish dummy streams (see Dummy Streams) from others. Other distinctions are usually made using representations

- `StreamsFamily V`

  All streams lie in the `StreamsFamily`

  ## 10.2 Operations applicable to All Streams

- `CloseStream(` stream`) O`

  In order to preserve system resources and to flush output streams every stream should be closed as soon as it is no longer used using `CloseStream`.

  It is an error to try to read characters from or write characters to a closed stream. Closing a stream tells the GAP kernel and/or the operating system kernel that the file is no longer needed. This may be necessary because the GAP kernel and/or the operating system may impose a limit on how many streams may be open simultaneously.

- `FileDescriptorOfStream(` stream`) O`

  returns the UNIX file descriptor of the underlying file. This is mainly useful for the `UNIXSelect` function call (see UNIXSelect). This is as of now only available on UNIX-like operating systems and only for streams to local processes and local files.

- `UNIXSelect(` inlist`,` outlist`,` exclist`,` timeoutsec`,` timeoutusec`) F`

  makes the UNIX C-library function `select` accessible from GAP for streams. The functionality is as described in the man page (see `man select`). The first three arguments must be lists containing UNIX file descriptors (integers) for streams. They can be obtained via `FileDescriptorOfStream` (see FileDescriptorOfStream) for streams to local processes and to local files. The argument timeoutsec is a timeout in seconds as in the `struct timeval` on the C level. The argument timeoutusec is analogously in microseconds. The total timeout is the sum of both. If one of those timeout arguments is not a small integer then no timeout is applicable (`fail` is allowed for the timeout arguments).

  The return value is the number of streams that are ready, this may be 0 if a timeout was specified. All file descriptors in the three lists that are not yet ready are replaced by `fail` in this function. So the lists are changed!

  This function is not available on the Macintosh architecture and is only available if your operating system has `select`, which is detected during compilation of GAP.

  ## 10.3 Operations for Input Streams

  Three operations normally used to read files: `Read`, `ReadAsFunction` and `ReadTest` can also be used to read GAP input from a stream. The input is immediately parsed and executed. When reading from a stream str, the GAP kernel generates calls to `ReadLine(`str`)` to supply text to the parser.

  Three further operations: `ReadByte`, `ReadLine` and `ReadAll`, support reading characters from an input stream without parsing them. This can be used to read data in any format and process it in GAP.

  Additional operations for input streams support detection of end of stream, and (for those streams for which it is appropriate) random access to the data.

- `Read(` input-text-stream`) O`

  reads the input-text-stream as input until `end-of-stream` occurs. See File Operations for details.

- `ReadAsFunction(` input-text-stream`) O`

  reads the input-text-stream as function and returns this function. See File Operations for details.

- `ReadTest(` input-text-stream`) O`

  reads the input-text-stream as test input until `end-of-stream` occurs. See File Operations for details.

  Example

      gap> # a function with local `a' does not change the global one
      gap> a := 1;;
      gap> i := InputTextString( "local a; a := 10; return a*10;" );;
      gap> ReadAsFunction(i)();
      100
      gap> a;
      1

      gap> # reading it via `Read' does
      gap> i := InputTextString( "a := 10;" );;
      gap> Read(i);
      gap> a;
      10

- `ReadByte(` input-stream`) O`

  `ReadByte` returns one character (returned as integer) from the input stream stream-in. `ReadByte` returns `fail` if there is no character available, in particular if it is at the end of a file.

  If stream-in is the input stream of a input/output process, `ReadByte` may also return `fail` if no byte is currently available.

  `ReadByte` is the basic operation for input streams. If a `ReadByte` method is installed for a user-defined type of stream which does not block, then all the other input stream operations will work (although possibly not at peak efficiency).

  `ReadByte` will wait (block) until a byte is available. For instance if the stream is a connection to another process, it will wait for the process to output a byte.

- `ReadLine(` input-stream`) O`

  `ReadLine` returns one line (returned as string with the newline) from the input stream input-stream. `ReadLine` reads in the input until a newline is read or the end-of-stream is encountered.

  If input-stream is the input stream of a input/output process, `ReadLine` may also return `fail` or return an incomplete line if the other process has not yet written any more. It will always wait (block) for at least one byte to be available, but will then return as much input as is available, up to a limit of one line

  A default method is supplied for `ReadLine` which simply calls `ReadByte` repeatedly. This is only safe for streams that cannot block. The kernel uses calls to `ReadLine` to supply input to the parser when reading from a stream.

- `ReadAll(` input-stream`) O`
- `ReadAll(` input-stream`,` limit`) O`

  `ReadAll` returns all characters as string from the input stream stream-in. It waits (blocks) until at least one character is available from the stream, or until there is evidence that no characters will ever be available again. This last indicates that the stream is at end-of-stream. Otherwise, it reads as much input as it can from the stream without blocking further and returns it to the user. If the stream is already at end of file, so that no bytes are available, `fail` is returned. In the case of a file stream connected to a normal file (not a pseudo-tty or named pipe or similar), all the bytes should be immediately available and this function will read the remainder of the file.

  With a second argument, at most limit bytes will be returned. Depending on the stream a bounded number of additional bytes may have been read into an internal buffer.

  A default method is supplied for `ReadAll` which simply calls `ReadLine` repeatedly. This is only really safe for streams which cannot block. Other streams should install a method for ReadAll

  Example

      gap> i := InputTextString( "1Hallo\nYou\n1" );;
      gap> ReadByte(i);
      49
      gap> CHAR_INT(last);
      '1'
      gap> ReadLine(i);
      "Hallo\n"
      gap> ReadLine(i);
      "You\n"
      gap> ReadLine(i);
      "1"
      gap> ReadLine(i);
      fail
      gap> ReadAll(i);
      ""
      gap> RewindStream(i);;
      gap> ReadAll(i);
      "1Hallo\nYou\n1"

- `IsEndOfStream(` input-stream`) O`

  `IsEndOfStream` returns `true` if the input stream is at end-of-stream, and `false` otherwise. Note that `IsEndOfStream` might return `false` even if the next `ReadByte` fails.

- `PositionStream(` input-stream`) O`

  Some input streams, such as string streams and file streams attached to disk files, support a form of random access by way of the operations `PositionStream`, `SeekPositionStream` and `RewindStream`. `PositionStream` returns a non-negative integer denoting the current position in the stream (usually the number of characters before the next one to be read.

  If this is not possible, for example for an input stream attached to standard input (normally the keyboard), then `fail` is returned

- `RewindStream(` input-stream`) O`

  `RewindStream` attempts to return an input stream to its starting condition, so that all the same characters can be read again. It returns `true` if the rewind succeeds and `fail` otherwise

  A default method implements RewindStream using `SeekPositionStream`.

- `SeekPositionStream(` input-stream`,` pos`) O`

  `SeekPositionStream` attempts to rewind or wind forward an input stream to the specified position. This is not possible for all streams. It returns `true` if the seek is successful and `fail` otherwise.

  ## 10.4 Operations for Output Streams

- `WriteByte(` output-stream`,` byte`) O`

  writes the next character (given as integer) to the output stream output-stream. The function returns `true` if the write succeeds and `fail` otherwise.

  `WriteByte` is the basic operation for output streams. If a `WriteByte` method is installed for a user-defined type of stream, then all the other output stream operations will work (although possibly not at peak efficiency).

- `WriteLine(` output-stream`,` string`) O`

  appends string to output-stream. A final newline is written. The function returns `true` if the write succeeds and `fail` otherwise.

  A default method is installed which implements `WriteLine` by repeated calls to `WriteByte`.

- `WriteAll(` output-stream`,` string`) O`

  appends string to output-stream. No final newline is written. The function returns `true` if the write succeeds and `fail` otherwise. It will block as long as necessary for the write operation to complete (for example for a child process to clear its input buffer )

  A default method is installed which implements `WriteAll` by repeated calls to `WriteByte`.

  When printing or appending to a stream (using `PrintTo`, or `AppendTo` or when logging to a stream), the kernel generates a call to `WriteAll` for each line output.

  Example

      gap> str := "";; a := OutputTextString(str,true);;
      gap> WriteByte(a,INT_CHAR('H'));
      true
      gap> WriteLine(a,"allo");
      true
      gap> WriteAll(a,"You\n");
      true
      gap> CloseStream(a);
      gap> Print(str);
      Hallo
      You

- `PrintTo(` output-stream`,` arg1`, ... ) F`
- `AppendTo(` output-stream`,` arg1`, ... ) F`

  These functions work like `Print`, except that the output is appended to the output stream output-stream.

  Example

      gap> str := "";; a := OutputTextString(str,true);;
      gap> AppendTo( a, (1,2,3), ":", Z(3) );
      gap> CloseStream(a);
      gap> Print( str, "\n" );
      (1,2,3):Z(3)

- `LogTo(` stream`) O`

  causes the subsequent interaction to be logged to the output stream stream. It works in precisely the same way as it does for files (see LogTo).

- `InputLogTo(` stream`) O`

  causes the subsequent input to be logged to the output stream stream. It works just like it does for files (see InputLogTo).

- `OutputLogTo(` stream`) O`

  causes the subsequent output to be logged to the output stream stream. It works just like it does for files (see OutputLogTo).

  When text is being sent to an output text stream via `PrintTo`, `AppendTo`, `LogTo`, etc., it is, by default formatted just as it would be were it being printed to the screen. Thus, it is broken into lines of reasonable length at (where possible) sensible places, lines containing elements of lists or records are indented, and so forth. This is appropriate if the output is eventually to be viewed by a human, and harmless if it to passed as input to GAP, but may be unhelpful if the output is to be passed as input to another program. It is possible to turn off this behaviour for a stream using the `SetPrintFormattingStatus` operation, and to test whether it is on or off using `PrintFormattingStatus`.

- `SetPrintFormattingStatus(` stream`,` newstatus`) O`

  sets whether output sent to the output stream stream via `PrintTo`, `AppendTo`, etc. (but not `WriteByte`, `WriteLine` or `WriteAll`) will be formatted with line breaks and indentation. If the second argument newstatus is `true` then output will be so formatted, and if `false` then it will not. If the stream is not a text stream, only `false` is allowed.

- `PrintFormattingStatus(` stream`) O`

  returns `true` if output sent to the output text stream stream via `PrintTo`, `AppendTo`, etc. (but not `WriteByte`, `WriteLine` or `WriteAll`) will be formatted with line breaks and indentation, and `false` otherwise (see SetPrintFormattingStatus). For non-text streams, it returns `false`.

  Example

      gap> s := "";; str := OutputTextString(s,false);;
      gap> PrintTo(str,Primes{[1..30]});
      gap> s;
      "[ 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71,\
       \n  73, 79, 83, 89, 97, 101, 103, 107, 109, 113 ]"
      gap> Print(s,"\n");
      [ 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71,
        73, 79, 83, 89, 97, 101, 103, 107, 109, 113 ]
      gap> SetPrintFormattingStatus(str, false);
      gap> PrintTo(str,Primes{[1..30]});
      gap> s;
      "[ 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71,\
       \n  73, 79, 83, 89, 97, 101, 103, 107, 109, 113 ][ 2, 3, 5, 7, 11, 13, 17, 19\
      , 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97, 101, 103\
      , 107, 109, 113 ]"
      gap> Print(s,"\n");
      [ 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71,
        73, 79, 83, 89, 97, 101, 103, 107, 109, 113 ][ 2, 3, 5, 7, 11, 13, 17, 19, 2\
      3, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97, 101, 103, 1\
      07, 109, 113 ]

  ## 10.5 File Streams

  File streams are streams associated with files. An input file stream reads the characters it delivers from a file, an output file stream prints the characters it receives to a file. The following functions can be used to create such streams. They return `fail` if an error occurred, in this case `LastSystemError` (see LastSystemError) can be used to get information about the error.

- `InputTextFile(` name-file`) O`

  `InputTextFile(` name-file`)` returns an input stream in the category `IsInputTextStream` that delivers the characters from the file name-file.

- `OutputTextFile(` name-file`,` append`) O`

  `OutputTextFile(` name-file`,` append`)` returns an output stream in the category `IsOutputTextFile` that writes received characters to the file name-file. If append is `false`, then the file is emptied first, otherwise received characters are added at the end of the list.

  Example

      gap> # use a temporary directory
      gap> name := Filename( DirectoryTemporary(), "test" );;
      gap> # create an output stream, append output, and close again
      gap> output := OutputTextFile( name, true );;
      gap> AppendTo( output, "Hallo\n", "You\n" );
      gap> CloseStream(output);
      gap> # create an input, print complete contents of file, and close
      gap> input := InputTextFile(name);;
      gap> Print( ReadAll(input) );
      Hallo
      You
      gap> CloseStream(input);
      gap> # append a single line
      gap> output := OutputTextFile( name, true );;
      gap> AppendTo( output, "AppendLine\n" );
      gap> # close output stream to flush the output
      gap> CloseStream(output);
      gap> # create an input, print complete contents of file, and close
      gap> input := InputTextFile(name);;
      gap> Print( ReadAll(input) );
      Hallo
      You
      AppendLine
      gap> CloseStream(input);

  ## 10.6 User Streams

  The following two commands create streams which accept characters from, or deliver characters to, the user, via the keyboard or the GAP session display.

- `InputTextUser( ) F`

  returns an input text stream which delivers characters typed by the user (or from the standard input device if it has been redirected). In normal circumstances, characters are delivered one by one as they are typed, without waiting until the end of a line. No prompts are printed.

- `OutputTextUser( ) F`

  returns an output stream which delivers characters to the user's display (or the standard output device if it has been redirected). Each character is delivered immediately it is written, without waiting for a full line of output. Text written in this way is not written to the session log (see LogTo).

  ## 10.7 String Streams

  String streams are streams associated with strings. An input string stream reads the characters it delivers from a string, an output string stream appends the characters it receives to a string. The following functions can be used to create such streams.

- `InputTextString(` string`) O`

  `InputTextString(` string`)`returns an input stream that delivers the characters from the string string. The string is not changed when reading characters from it and changing the string after the call to `InputTextString` has no influence on the input stream.

- `OutputTextString(` list`,` append`) O`

  returns an output stream that puts all received characters into the list list. If append is `false`, then the list is emptied first, otherwise received characters are added at the end of the list.

  Example

      gap> # read input from a string
      gap> input := InputTextString( "Hallo\nYou\n" );;
      gap> ReadLine(input);
      "Hallo\n"
      gap> ReadLine(input);
      "You\n"
      gap> # print to a string
      gap> str := "";;
      gap> out := OutputTextString( str, true );;
      gap> PrintTo( out, 1, "\n", (1,2,3,4)(5,6), "\n" );
      gap> CloseStream(out);
      gap> Print( str );
      1
      (1,2,3,4)(5,6)

  ## 10.8 Input-Output Streams

  Input-output streams capture bidirectional communications between GAP and another process, either locally or (@as yet unimplemented@) remotely.

  Such streams support the basic operations of both input and output streams. They should provide some buffering, allowing output data to be written to the stream, even when input data is waiting to be read, but the amount of this buffering is operating system dependent, and the user should take care not to get too far ahead in writing, or behind in reading, or deadlock may occur.

- `IsInputOutputStream(` obj`) C`

  `IsInputOutputStream` is the Category of Input-Output Streams; it returns `true` if the obj is an input-output stream and `false` otherwise.

  At present the only type of Input-Output streams that are implemented provide communication with a local child process, using a pseudo-tty.

  Like other streams, write operations are blocking, read operations will block to get the first character, but not thereafter.

  As far as possible, no translation is done on characters written to, or read from the stream, and no control characters have special effects, but the details of particular pseudo-tty implementations may effect this.

- `InputOutputLocalProcess(` dir`,` executable`,` args`) F`

  starts up a slave process, whose executable file is executable, with ``command line'' arguments args in the directory dir. (Suitable choices for dir are `DirectoryCurrent()` or `DirectoryTemporary()` (see Section Directories); `DirectoryTemporary()` may be a good choice when executable generates output files that it doesn't itself remove afterwards.) `InputOutputLocalProcess` returns an InputOutputStream object. Bytes written to this stream are received by the slave process as if typed at a terminal on standard input. Bytes written to standard output by the slave process can be read from the stream.

  When the stream is closed, the signal SIGTERM is delivered to the child process, which is expected to exit.

      gap> d := DirectoryCurrent();
      dir("./")
      gap> f := Filename(DirectoriesSystemPrograms(), "rev");
      "/usr/bin/rev"
      gap> s := InputOutputLocalProcess(d,f,[]);
      < input/output stream to rev >
      gap> WriteLine(s,"The cat sat on the mat");
      true
      gap> Print(ReadLine(s));
      tam eht no tas tac ehT
      gap> x := ListWithIdenticalEntries(10000,'x');;
      gap> ConvertToStringRep(x);
      gap> WriteLine(s,x);
      true
      gap> WriteByte(s,INT_CHAR('\n'));
      true
      gap> y := ReadAll(s);;
      gap> Length(y);
      4096
      gap> CloseStream(s);
      gap> s;
      < closed input/output stream to rev >

- `ReadAllLine(` iostream`[,` nofail`][,` IsAllLine`] ) O`

  For an input/output stream iostream `ReadAllLine` reads until a newline character if any input is found or returns `fail` if no input is found, i.e. if any input is found `ReadAllLine` is non-blocking.

  If the argument nofail (which must be `false` or `true`) is provided and it is set to `true` then `ReadAllLine` will wait, if necessary, for input and never return `fail`.

  If the argument IsAllLine (which must be a function that takes a string argument and returns either `true` or `false`) then it is used to determine what constitutes a whole line. The default behaviour is equivalent to passing the function

      line -> 0 < Length(line) and line[Length(line)] = '\n'

  for the IsAllLine argument. The purpose of the IsAllLine argument is to cater for the case where the input being read is from an external process that writes a ``prompt'' for data that does not terminate with a newline.

  If the first argument is an input stream but not an input/output stream then `ReadAllLine` behaves as if `ReadLine` was called with just the first argument and any additional arguments are ignored.

  ## 10.9 Dummy Streams

  The following two commands create dummy streams which will consume all characters and never deliver one.

- `InputTextNone( ) F`

  returns a dummy input text stream, which delivers no characters, i.e., it is always at end of stream. Its main use is for calls to `Process` (see Process) when the started program does not read anything.

- `OutputTextNone( ) F`

  returns a dummy output stream, which discards all received characters. Its main use is for calls to `Process` (see Process) when the started program does not write anything.

  ## 10.10 Handling of Streams in the Background

  This section describes a feature of the GAP kernel that can be used to handle pending streams somehow ``in the background''. This is currently not available on the Macintosh architecture and only on operating systems that have `select`.

  Right before GAP reads a keypress from the keyboard it calls a little subroutine that can handle streams that are ready to be read or ready to be written. This means that GAP can handle these streams during user input on the command line. Note that this does not work when GAP is in the middle of some calculation.

  This feature is used in the following way. One can install handler functions for reading or writing streams. This is done via:

- `InstallCharReadHookFunc(` stream`,` mode`,` func`) F`

  installs the function func as a handler function for the stream stream. The argument mode decides, for what operations on the stream this function is installed. mode must be a string, in which a letter `r` means ``read'', `w` means ``write'' and `x` means ``exception'', according to the `select` function call in the UNIX C-library (see `man select` and UNIXSelect). More than one letter is allowed in mode. As described above the function is called in a situation when GAP is reading a character from the keyboard. Handler functions should not use much time to complete.

  This functionality does not work on the Macintosh architecture and only works if the operating system has a `select` function.

  Handlers can be removed via:

- `UnInstallCharReadHookFunc(` stream`,` func`) F`

  uninstalls the function func as a handler function for the stream stream. All instances are deinstalled, regardless of the mode of operation (read, write, exception).

  This functionality does not work on the Macintosh architecture and only works if the operating system has a `select` function.

  Note that handler functions must not return anything and get one integer argument, which refers to an index in one of the following arrays (according to whether the function was installed for input, output or exceptions on the stream). Handler functions usually should not output anything on the standard output because this ruins the command line during command line editing.

  [Top] [Up] [Previous] [Next] [Index]

  GAP 4 manual
  October 2007