[Go to first, previous, next page;   contents;   index]

Chapter 11

Input and Output

11.1  Ports

By definition, ports in MzScheme produce and consume bytes. When a port is provided to a character-based operation, such as read, the port's bytes are read and interpreted as a UTF-8 encoding of characters (see also section 1.2.3). Thus, reading a single character may require reading multiple bytes, and a procedure like char-ready? may need to peek several bytes into the stream to determine whether a character is available. In the case of a byte stream that does not correspond to a valid UTF-8 encoding, functions such as read-char may need to peek one byte ahead in the stream to discover that the stream is not a valid encoding.

When an input port produces a sequence of bytes that is not a valid UTF-8 encoding in a character-reading context, then bytes that constitute an invalid sequence are converted to the character ``?''. Specifically, bytes 255 and 254 are always converted to ``?'', bytes in the range 192 to 253 produce ``?'' when they are not followed by bytes that form a valid UTF-8 encoding, and bytes in the range 128 to 191 are converted to ``?'' when they are not part of a valid encoding that was started by a preceding byte in the range 192 to 253. To put it another way, when reading a sequence of bytes as characters, a minimal set of bytes are changed to 6325 so that the entire sequence of bytes is a valid UTF-8 encoding.

See section 3.6 for procedures that facilitate conversions using UTF-8 or other encodings. See also reencode-input-port and reencode-output-port in Chapter 33 in PLT MzLib: Libraries Manual for obtaining a UTF-8-based port from one that uses a different encoding of characters.

(port? v) returns #t if either either (input-port? v) or (output-port? v) is #t, #f otherwise.

(file-stream-port? port) returns #t if the given port is a file-stream port (see section 11.1.6, #f otherwise.

(terminal-port? port) returns #t if the given port is attached to an interactive terminal, #f otherwise.

11.1.1  End-of-File Constant

The global variable eof is bound to the end-of-file value. The standard Scheme predicate eof-object? returns #t only when applied to this value.

11.1.2  Current Ports

The standard Scheme procedures current-input-port and current-output-port are implemented as parameters in MzScheme. See section 7.9.1.2 for more information.

11.1.3  Opening File Ports

The open-input-file and open-output-file procedures accept an optional flag argument after the filename that specifies a mode for the file:

The open-output-file procedure can also take a flag argument that specifies how to proceed when a file with the specified name already exists:

The open-input-output-file procedure takes the same arguments as open-output-file, but it produces two values: an input port and an output port. The two ports are connected in that they share the underlying file device. This procedure is intended for use with special devices that can be opened by only one process, such as COM1 in Windows. For regular files, sharing the device can be confusing. For example, using one port does not automatically flush the other port's buffer (see section 11.1.6 for more information about buffers), and reading or writing in one port moves the file position (if any) for the other port. For regular files, use separate open-input-file and open-output-file calls to avoid confusion.

Extra flag arguments are passed to open-output-file in any order. Appropriate flag arguments can also be passed as the last argument(s) to call-with-input-file, with-input-from-file, call-with-output-file, and with-output-to-file. When conflicting flag arguments (e.g., both 'error and 'replace) are provided to open-output-file, with-output-to-file, or call-with-output-file, the exn:fail:contract exception is raised.

Both with-input-from-file and with-output-to-file close the port they create if control jumps out of the supplied thunk (either through a continuation or an exception), and the port remains closed if control jumps back into the thunk. The current input or output port is installed and restored with parameterize (see section 7.9.2).

See section 11.1.6 for more information on file ports. When an input or output file-stream port is created, it is placed into the management of the current custodian (see section 9.2).

11.1.4  Pipes

(make-pipe [limit-k input-name-v output-name-v]) returns two port values (see section 2.2): the first port is an input port and the second is an output port. Data written to the output port is read from the input port. The ports do not need to be explicitly closed.

The optional limit-k argument can be #f or a positive exact integer. If limit-k is omitted or #f, the new pipe holds an unlimited number of unread bytes (i.e., limited only by the available memory). If limit-k is a positive number, then the pipe will hold at most limit-k unread/unpeeked bytes; writing to the pipe's output port thereafter will block until a read or peek from the input port makes more space available. (Peeks effectively extend the port's capacity until the peeked bytes are read.)

The optional input-name-v and output-name-v are used as the names for the returned input and out ports, respectively, if they are supplied. Otherwise, the name of each port is 'pipe.

(pipe-content-length pipe-port) returns the number of bytes contained in a pipe, where pipe-port is either of the pipe's ports produced by make-pipe. The pipe's content length counts all bytes that have been written to the pipe and not yet read (though possibly peeked).

11.1.5  String Ports

Scheme input and output can be read from or collected into a string or byte string:

String input and output ports do not need to be explicitly closed. The file-position procedure, described in section 11.1.6, works for string ports in position-setting mode.

Example: 

(define i (open-input-string "hello world"))
(define o (open-output-string))
(write (read i) o)
(get-output-string o) ; => "hello"

11.1.6  File-Stream Ports

A port created by open-input-file, open-output-file, subprocess, and related functions is a file-stream port. The initial input, output, and error ports in stand-alone MzScheme are also file-stream ports. The file-stream-port? predicate recognizes file-stream ports.

An input port is blocked buffered by default, which means that on any read, the buffer is filled with immediately-available bytes to speed up future reads. Thus, if a file is modified between a pair of reads to the file, the second read can produce stale data. Calling file-position to set an input port's file position flushes its buffer.

Most output ports are block buffered by default, but a terminal output port is line buffered, and the error output port is unbuffered. An output buffer is filled with a sequence of written bytes to be committed as a group, either when the buffer is full (in block mode) or when a newline is written (in line mode).

A port's buffering can be changed via file-stream-buffer-mode (described below). The two ports produced by open-input-output-file have independent buffers.

The following procedures work primarily on file-stream ports:

11.1.7  Custom Ports

The make-input-port and make-output-port procedures create custom ports with arbitrary control procedures. Correctly implementing a custom port can be tricky, because it amounts to implementing a device driver. Custom ports are mainly useful to obtain fine control over the action of committing bytes as read or written.

Many simple port variations can be implemented using threads and pipes. For example, if get-next-char is a function that produces either a character or eof, it can be turned into an input port as follows 

(let-values ([(r w) (make-pipe 4096)])
  ;; Create a thread to move chars from get-next-char to the pipe
  (thread (lambda () (let loop ()
                       (let ([v (get-next-char)])
                         (if (eof? v)
                             (close-output-port w)
                             (begin
                               (write-char v f)
                               (loop)))))))
   ;; Return the read end of the pipe
   r)

The port.ss in MzLib provides several other port constructors; see Chapter 33 in PLT MzLib: Libraries Manual.

11.1.7.1  Custom Input

(make-input-port name-v read-proc optional-peek-proc close-proc [optional-progress-evt-proc optional-commit-proc optional-location-proc count-lines!-proc init-position optional-buffer-mode-proc]) creates an input port. The port is immediately open for reading. If close-proc procedure has no side effects, then the port need not be explicitly closed.

When read-proc or optional-peek-proc (or an event produced by one of these) returns a procedure, and the procedure is used to obtain a non-byte result.27 The procedure is called by read,28 read-syntax, read-honu, read-honu-syntax, read-byte-or-special, read-char-or-special, peek-byte-or-special, or peek-char-or-special. The special-value procedure can return an arbitrary value, and it will be called zero or one times (not necessarily before further reads or peeks from the port). See section 11.2.9 for more details on the procedure's arguments and result.

If read-proc or optional-peek-proc returns a special procedure when called by any reading procedure other than read, read-syntax, read-honu, read-honu-syntax, read-char-or-special, peek-char-or-special, read-byte-or-special, or peek-byte-or-special, then the exn:fail:contract exception is raised.

Examples: 

;; A port with no input...
;; Easy: (open-input-bytes #"")
;; Hard:
(define /dev/null-in 
  (make-input-port 'null
                   (lambda (s) eof)
                   (lambda (skip s progress-evt) eof)
                   void
                   (lambda () never-evt)
                   (lambda (k progress-evt done-evt)
                     (error "no successful peeks!"))))
(read-char /dev/null-in) ; => eof
(peek-char /dev/null-in) ; => eof
(read-byte-or-special /dev/null-in)     ; => eof
(peek-byte-or-special /dev/null-in 100) ; => eof

;; A port that produces a stream of 1s:
(define infinite-ones 
  (make-input-port
   'ones
   (lambda (s) 
     (bytes-set! s 0 (char->integer #\1)) 1)
   #f
   void))
(read-string 5 infinite-ones) ; => "11111"

;; But we can't peek ahead arbitrarily far, because the
;; automatic peek must record the skipped bytes:
(peek-string 5 (expt 2 5000) infinite-ones) ; => error: out of memory

;; An infinite stream of 1s with a specific peek procedure:
(define infinite-ones 
  (let ([one! (lambda (s) 
                (bytes-set! s 0 (char->integer #\1)) 1)])
    (make-input-port
     'ones
     one!
     (lambda (s skip progress-evt) (one! s))
     void)))
(read-string 5 infinite-ones) ; => "11111"

;; Now we can peek ahead arbitrarily far:
(peek-string 5 (expt 2 5000) infinite-ones) ; => "11111"

;; The port doesn't supply procedures to implement progress events:
(port-provides-progress-evts? infinite-ones) ; => #f
(port-progress-evt infinite-ones) ; error: no progress events

;; Non-byte port results:
(define infinite-voids
  (make-input-port
   'voids
   (lambda (s) (lambda args 'void))
   (lambda (skip s) (lambda args 'void))
   void))
(read-char infinite-voids) ; => error: non-char in an unsupported context
(read-char-or-special infinite-voids) ; => 'void

;; This port produces 0, 1, 2, 0, 1, 2, etc., but it is not
;; thread-safe, because multiple threads might read and change n.
(define mod3-cycle/one-thread
  (let* ([n 2]
         [mod! (lambda (s delta)
                 (bytes-set! s 0 (+ 48 (modulo (+ n delta) 3)))
                 1)])
    (make-input-port
     'mod3-cycle/not-thread-safe
     (lambda (s) 
       (set! n (modulo (add1 n) 3))
       (mod! s 0))
     (lambda (s skip) 
       (mod! s skip))
     void)))
(read-string 5 mod3-cycle/one-thread) ; => "01201"
(peek-string 5 (expt 2 5000) mod3-cycle/one-thread) ; => "20120"

;; Same thing, but thread-safe and kill-safe, and with progress
;; events. Only the server thread touches the stateful part
;; directly. (See the output port examples for a simpler thread-safe
;; example, but this one is more general.)
(define (make-mod3-cycle)
  (define read-req-ch (make-channel))
  (define peek-req-ch (make-channel))
  (define progress-req-ch (make-channel))
  (define commit-req-ch (make-channel))
  (define close-req-ch (make-channel))
  (define closed? #f)
  (define n 0)
  (define progress-sema #f)
  (define (mod! s delta)
    (bytes-set! s 0 (+ 48 (modulo (+ n delta) 3)))
    1)
  ;; ----------------------------------------
  ;; The server has a list of outstanding commit requests,
  ;;  and it also must service each port operation (read, 
  ;;  progress-evt, etc.)
  (define (serve commit-reqs response-evts)
    (apply
     sync
     (handle-evt read-req-ch (handle-read commit-reqs response-evts))
     (handle-evt progress-req-ch (handle-progress commit-reqs response-evts))
     (handle-evt commit-req-ch (add-commit commit-reqs response-evts))
     (handle-evt close-req-ch (handle-close commit-reqs response-evts))
     (append
      (map (make-handle-response commit-reqs response-evts) response-evts)
      (map (make-handle-commit commit-reqs response-evts) commit-reqs))))
  ;; Read/peek request: fill in the string and commit
  (define ((handle-read commit-reqs response-evts) r)
    (let ([s (car r)]
          [skip (cadr r)]
          [ch (caddr r)]
          [nack (cadddr r)]
          [peek? (cddddr r)])
      (unless closed?
        (mod! s skip)
        (unless peek?
          (commit! 1)))
      ;; Add an event to respond:
      (serve commit-reqs
             (cons (choice-evt nack
                               (channel-put-evt ch (if closed? 0 1)))
                   response-evts))))
  ;; Progress request: send a peek evt for the current 
  ;;  progress-sema
  (define ((handle-progress commit-reqs response-evts) r)
    (let ([ch (car r)]
          [nack (cdr r)])
      (unless progress-sema
        (set! progress-sema (make-semaphore (if closed? 1 0))))
      ;; Add an event to respond:
      (serve commit-reqs
             (cons (choice-evt nack
                               (channel-put-evt
                                ch
                                (semaphore-peek-evt progress-sema)))
                   response-evts))))
  ;; Commit request: add the request to the list
  (define ((add-commit commit-reqs response-evts) r)
    (serve (cons r commit-reqs) response-evts))
  ;; Commit handling: watch out for progress, in which case
  ;;  the response is a commit failure; otherwise, try
  ;;  to sync for a commit. In either event, remove the
  ;;  request from the list
  (define ((make-handle-commit commit-reqs response-evts) r)
    (let ([k (car r)]
          [progress-evt (cadr r)]
          [done-evt (caddr r)]
          [ch (cadddr r)]
          [nack (cddddr r)])
      ;; Note: we don't check that k is < the sum of
      ;;  previous peeks, because the entire stream is actually
      ;;  known, but we could send an exception in that case.
      (choice-evt
       (handle-evt progress-evt
                   (lambda (x) 
                     (sync nack (channel-put-evt ch #f))
                     (serve (remq r commit-reqs) response-evts)))
       ;; Only create an event to satisfy done-evt if progress-evt
       ;;  isn't already ready.
       ;; Afterward, if progress-evt becomes ready, then this
       ;;  event-making function will be called again, because
       ;;  the server controls all posts to progress-evt.
       (if (sync/timeout 0 progress-evt)
           never-evt
           (handle-evt done-evt
                       (lambda (v)
                         (commit! k)
                         (sync nack (channel-put-evt ch #t))
                         (serve (remq r commit-reqs) response-evts)))))))
  ;; Response handling: as soon as the respondee listens,
  ;;  remove the response
  (define ((make-handle-response commit-reqs response-evts) evt)
    (handle-evt evt
                (lambda (x)
                  (serve commit-reqs
                         (remq evt response-evts)))))
  ;; Close handling: post the progress sema, if any, and set
  ;;   the closed? flag
  (define ((handle-close commit-reqs response-evts) r)
    (let ([ch (car r)]
          [nack (cdr r)])
      (set! closed? #t)
      (when progress-sema
        (semaphore-post progress-sema))
      (serve commit-reqs
             (cons (choice-evt nack
                               (channel-put-evt ch (void)))
                   response-evts))))
  ;; Helper for reads and post-peek commits:
  (define (commit! k)
    (when progress-sema
      (semaphore-post progress-sema)
      (set! progress-sema #f))
    (set! n (+ n k)))
  ;; Start the server thread:
  (define server-thread (thread (lambda () (serve null null))))
  ;; ----------------------------------------
  ;; Client-side helpers:
  (define (req-evt f)
    (nack-guard-evt
     (lambda (nack)
       ;; Be sure that the server thread is running:
       (thread-resume server-thread (current-thread))
       ;; Create a channel to hold the reply:
       (let ([ch (make-channel)])
         (f ch nack)
         ch))))
  (define (read-or-peek-evt s skip peek?)
    (req-evt (lambda (ch nack)
               (channel-put read-req-ch (list* s skip ch nack peek?)))))
  ;; Make the port:
  (make-input-port 'mod3-cycle
                   ;; Each handler for the port just sends
                   ;;  a request to the server
                   (lambda (s) (read-or-peek-evt s 0 #f))
                   (lambda (s skip) (read-or-peek-evt s skip #t))
                   (lambda () ; close
                     (sync (req-evt
                            (lambda (ch nack)
                              (channel-put progress-req-ch (list* ch nack))))))
                   (lambda () ; progress-evt
                     (sync (req-evt
                            (lambda (ch nack)
                              (channel-put progress-req-ch (list* ch nack))))))
                   (lambda (k progress-evt done-evt)  ; commit
                     (sync (req-evt
                            (lambda (ch nack)
                              (channel-put commit-req-ch
                                           (list* k progress-evt done-evt ch nack))))))))

(let ([mod3-cycle (make-mod3-cycle)])
  (let ([result1 #f]
        [result2 #f])
    (let ([t1 (thread (lambda ()
                        (set! result1 (read-string 5 mod3-cycle))))]
          [t2 (thread (lambda ()
                        (set! result2 (read-string 5 mod3-cycle))))])
      (thread-wait t1)
      (thread-wait t2)
      (string-append result1 "," result2))) ; => "02120,10201", maybe
  (let ([s (make-bytes 1)]
        [progress-evt (port-progress-evt mod3-cycle)])
    (peek-bytes-avail! s 0 progress-evt mod3-cycle) ; => 1
    s                                    ; => #"1"
    (port-commit-peeked 1 progress-evt (make-semaphore 1)
                           mod3-cycle)   ; => #t
    (sync/timeout 0 progress-evt)        ; => progress-evt
    (peek-bytes-avail! s 0 progress-evt mod3-cycle) ; => 0
    (port-commit-peeked 1 progress-evt (make-semaphore 1) 
                           mod3-cycle))  ; => #f
  (close-input-port mod3-cycle))

11.1.7.2  Custom Output

(make-output-port name-v evt write-proc close-proc [optional-write-special-proc optional-write-evt-proc optional-special-evt-proc optional-location-proc count-lines!-proc init-position optional-buffer-mode-proc]) creates an output port. The port is immediately open for writing. If close-proc procedure has no side effects, then the port need not be explicitly closed. The port can buffer data within its write-proc and optional-write-special-proc procedures.

Examples: 

;; A port that writes anything to nowhere:
(define /dev/null-out
  (make-output-port 
   'null
   always-evt
   (lambda (s start end non-block? breakable?) (- end start))
   void
   (lambda (special non-block? breakable?) #t)
   (lambda (s start end) (wrap-evt
                          always-evt
                          (lambda (x)
                            (- end start))))
   (lambda (special) always-evt)))
(display "hello" /dev/null-out)            ; => void
(write-bytes-avail #"hello" /dev/null-out) ; => 5
(write-special 'hello /dev/null-out)       ; => #t
(sync (write-bytes-avail-evt #"hello" /dev/null-out)) ; => 5

;; A part that accumulates bytes as characters in a list,
;;  but not in a thread-safe way:
(define accum-list null)
(define accumulator/not-thread-safe
  (make-output-port 
   'accum/not-thread-safe
   always-evt
   (lambda (s start end non-block? breakable?)
     (set! accum-list
           (append accum-list
                   (map integer->char
                        (bytes->list (subbytes s start end)))))
     (- end start))
   void))
(display "hello" accumulator/not-thread-safe)
accum-list ; => '(#\h #\e #\l #\l #\o)

;; Same as before, but with simple thread-safety:
(define accum-list null)
(define accumulator 
  (let* ([lock (make-semaphore 1)]
         [lock-peek-evt (semaphore-peek-evt lock)])
    (make-output-port
     'accum
     lock-peek-evt
     (lambda (s start end non-block? breakable?)
       (if (semaphore-try-wait? lock)
           (begin
             (set! accum-list
                   (append accum-list
                           (map integer->char
                                (bytes->list (subbytes s start end)))))
             (semaphore-post lock)
             (- end start))
           ;; Cheap strategy: block until the list is unlocked,
           ;;   then return 0, so we get called again
           (wrap-evt
            lock-peek
            (lambda (x) 0))))
     void)))
(display "hello" accumulator)
accum-list ; => '(#\h #\e #\l #\l #\o)

;; A port that transforms data before sending it on
;;  to another port. Atomic writes exploit the
;;  underlying port's ability for atomic writes.
(define (make-latin-1-capitalize port)
  (define (byte-upcase s start end)
    (list->bytes
     (map (lambda (b) (char->integer
                       (char-upcase
                        (integer->char b))))
          (bytes->list (subbytes s start end)))))
  (make-output-port
   'byte-upcase
   ;; This port is ready when the original is ready:
   port
   ;; Writing procedure:
   (lambda (s start end non-block? breakable?)
     (let ([s (byte-upcase s start end)])
       (if non-block?
           (write-bytes-avail* s port)
           (begin
             (display s port)
             (bytes-length s)))))
   ;; Close procedure --- close original port:
   (lambda () (close-output-port port))
   #f
   ;; Write event:
   (and (port-writes-atomic? port)
        (lambda (s start end)
          (write-bytes-avail-evt (byte-upcase s start end) port)))))
(define orig-port (open-output-string))
(define cap-port (make-latin-1-capitalize orig-port))
(display "Hello" cap-port)
(get-output-string orig-port) ; => "HELLO"
(sync (write-bytes-avail-evt #"Bye" cap-port)) ; => 3
(get-output-string orig-port) ; => "HELLOBYE"

11.2  Reading and Writing

MzScheme's support for reading and writing includes many extensions compared to R5RS, both at the level of individual bytes and characters and at the level of S-expressions.

11.2.1  Reading Bytes, Characters, and Strings

In addition to the standard reading procedures, MzScheme provides byte-reading procedure, block-reading procedures such as read-line, and more.