JavaScript File object proposal
by Henri Torgemane,
Michael Ang <mang@subcarrier.org>,
and Vladimir Livshits <val4@cornell.edu>

This documents describes a proposed API for the JavaScript File object. A lot of the API has been implemented (but not tested), and there is work yet to be done. Please send comments to norris@netscape.com.

Many scripts in JavaScript require I/O capabilities and particularly access to normal files. This document describes such a File object that can be implemented by host implementations.

The interface is a mix of the Server Side JS File object and the Java File object and tries to follow existing implicit JavaScript usage ( readln instead of readLine, for example ).

The File object gives two ways to access the data inside a file: a text oriented access, based on characters, and a binary oriented access, based on bytes. In the text mode, maximum string length is currently 256. Different encodings are allowed, with ASCII, UTF8 (unicode), and UCS2 (binary) currently supported.

The implementation is currenly based mostly on NSPR PRFileDesc, (sorry, no public link for this yet) but there is also a back door that allows one to create files based on standard C FILE type. This is used to give the ability to initialize JavaScript files based on standard streams stdin, stdout, and stderr. This is also how pipes are implemented. With FILE-style files, however, not all operations are uniformly supported everywhere, after all it is the goal of NSPR to hide the discrepancies among different platforms. 

0.0 Changelog

• 1/18/98 - volodya added lots of little changes to the API to document the features that were either added or fixed. Added description of native file handling, how the objects shout react in different cases, etc. In particular, see status for most current info.
• 12/18/98 - mang is taking some time off from being a mozilla.organ. The interim owner is Norris Boyd.
• added nsFileSpec email thread. The File object should use nsFileSpec like functionality instead of it's current filename manipulation code.
• added properties append, autoFlush, and replace. These properties replace the comma-separated mode, which has been discarded. From mccabe@netscape.com.
• added more deep thoughts to Unresolved questions/TODO
• 12/09/98 - parent property is now set to null if the file has no parent, instead of containing the file itself. From fur@netscape.com.
• 12/07/98 - in and delete are reserved identifiers. Changed in to input, out to output, err to error, and delete to remove. From brendan@netscape.com.

1.0 Filenames

Filenames are specified as strings that have an implementation defined format. Use of standard "file:" URLs is encouraged.

Examples of possible prefix strings are "/" to indicate the Unix root directory, "c:" to specify a Windows drive letter, and "file:" to indicate a file URL.

When a file is constructed, leading and trailing spaces are removed from the filename, so new File(" abc.txt    ") just creates a file called abc.txt. Filenames starting and ending with '|' are interpreted as pipes a la Perl.

We should be able to do things like this:

var mail = new File("|/usr/bin/mail foo@bar.com");
mail.writeln("I love JavaScript.\nPipe support is especially good!");
mail.close();

2.0 Properties of the File constructor

 

File	File constructor method
input	a File object that represents the standard input.
output	a File object that represents the standard output.
error	a File object that represents the standard error.
currentDir	a File object referring to the current directory; this property may be set.
separator	the system name separator.

Note: The first tree properties should probably be enabled only in certain settings, not in the browser context, for example. Keep in mind that some methods such as isFile, for example, will return an error if called on one of them.

2.1 Properties of File instances

 

name	contains the name of the file.
path	contains the canonical path to the file.
length	contains the number of characters in the file.
parent	contains the directory containing the file.
type	contains a string specifying the type of data or encoding contained in the file. Currently "ascii","utf8" (UTF8), or "ucs2" (UCS2).
mode	contains the file mode.
position	contains the current offset in the file; this property may be set.
isFile	true if the file is a normal file
isDirectory	true if the file is a directory.
exists	true if the file exists.
isOpen	true if the file has been successfully opened.
isNative	true for pipes and standard streams
lastModified	contains a Date object for the time the file was last modified.
canRead	true if the file can be read.
canWrite	true if the file can be written.
canAppend	contains a flag indicating if the file is in append mode.
canReplace	contains a flag indicating if the file is in replace mode.
hasRandomAccess	contains a flag indicating if the "position" property can be used.
hasAutoFlush	contains a flag indicating if the file is in automatic flush mode.

2.2 Instance methods

 

open(type,mode)	open a file
close()	close a file
flush()	flush the data waiting to be sent to a file
seek(numChars)	skip the next numChars characters
read(numChars)	read numChars characters and return them into a string
readln()	read a line of characters from the file
readAll()	read all the remaining lines in the file and return an array of strings
write(arg0,arg1,...,argn)	write the parameters as strings to the file
writeln(arg0,arg1,...,argn)	same as write, and add a new line separator for the current platform
writeAll(Array)	write an array of objects sequentially into a file.
remove()	removes a file or a directory
copyTo(filename)	copy the content of a file to another location
renameTo(filename)	moves/renames a file
list(filter)	get a list of files inside a directory.
mkdir()	create a new directory.
toString()	return the canonical path to the file.
toURL()	return the name of the file as a "file:" URL if possible.

3.0 Properties of the File constructor

File

File constructor. Expects filename as the only argument. The filename can be preceeded with "file://", which is stripped off. If no argument is supplied, current working directory is the file object that is returned.

input

A File object that represents the standard input stream. Initialized in certain settings.

output

A File object that represents the standard output stream. Initialized in certain settings

error

A File object that represents the standard error stream. Initialized in certain settings.

currentDir

This property contains a File object pointing to the current directory. Assigning to this property causes the current directory to change. The following syntax is supported:

File.currentDir = new File("/");
    which is functionally equivalent to
File.currentDir = "/";
separator

The system name separator. For example, "/" on Unix.

3.1 Properties of File instances

name

This property contains the name of the file. Defined as "Standard input(output/error) stream" for streams. Undefined for pipes.

path

This property contains the canonical path to the file. Defined as "Standard input(output/error) stream" for streams. Returns the path that was used to create the pipe for pipes. Initial and trailing spaces are removed.

length

For a directory, the number of files in it not counting the current directory and parent directory entries. For a file, this property contains the number of characters in the file. Undefined for pipes and standard streams.

parent

This property contains a File object pointing to the directory containing the file. If the file isn't contained in anything (root directory), the property is null. Undefined for pipes and standard streams.

type

This property contains a string specifying the type of data or encoding used when the file was opened. Specific text encodings supported are "ascii", "unicode", and "binary". Undefined for directories and closed files.

mode

This property contains the mode of the file. It can be somewhat different from what was passed to open if you have repeating attributes ("read,read") or if some attributes were ignored. Returns undefined if the file is closed.

position

This property contains the current offset in the file. this is the only property that may be set. This property is undefined if the file is not in random access mode or if it is a pipe or a standard stream. Also undefined for closed files. Setting this property performs a seek operation on the file.

isFile

This property contains true if the file is a regular data file. False for pipes and standard streams. Note that on some platforms, both "isDirectory" and "isFile" may return false for some types of files (for example, unix named pipes).

isDirectory

This property contains true if the file is a directory. False for pipes and standard streams.

exists

This property equals true if the file exists. False for pipes and standard streams.

isOpen

This property contains true if the file has been successfully opened and is still open. Keep in mind that standard streams and pipes are open by default and don't requre opening.

isNative

This property contains true for pipes and standard streams.

lastModified

This property contains a Date object for the time the file was last modified. Undefined for pipes and standard streams.

canRead

This property contains true if the file can be read. The mode with which the file was opened (if it was) is respected. true for File.input, false for other streams. true for pipes starting with '|', false otherwise.

canWrite

This property contains true if the file can be written. The mode with which the file was opened (if it was) is respected. true for File.output and File.error, false for File.input. true for pipes ending with '|', false otherwise.

canAppend

This property is true if the file was open for appending. False for pipes and standard streams. Undefined for closed files.

canReplace

This property is true if the file was open with replace flag enabled. False for pipes and standard streams. Undefined for closed files.

hasRandomAccess

This property contains a flag indicating that the file is being opened in random access mode. This means the "position" property can be read and set. False for pipes and standard streams. Undefined for closed files.

hasAutoFlush

This property contains true if the file is in automatic flush mode. If this is set to true, each call to writeln will flush the stream. False for pipes and standard streams. Undefined for closed files.

3.1.1 Special properties for directories

A File object that represents a directory gets additional properties that represent the files contained in the directory.

These properties have the same names as the files in the directory.
 

Example

myDir = new File("some/directory"); myFileInDir = myDir.foo; Named property lookup can be used to refer to files whose names are not valid as normal property names.  mySameFileInDir = myDir["foo"]; myOtherFile = myDir["some long filename with spaces and such"];

3.2 Instance methods

open(type,mode)

If no mode is specified, the default mode that is used for regular files is "read,append,create". Pipes are automatically open at creation time with mode defaulting to "read" or "write" depending on pipe type. If no type is specified, the default type is "text".
 

Valid modes (can be combined)
read	specify that the file is opened for reading.
write	specify that the file is opened for writing.
readWrite	specify that the file is opened for both reading and writing
randomAccess	specify that the file is opened for both reading and writing.
append	position the file pointer at the end of the file.
autoflush	ask to automatically flush the output when a newline character is written.
replace	erase the content of the file before opening it.

You can also use the format "create=yes, append=yes", etc. Please note that the case of letters is important.
 

Notes:
• if both "read" and "write" are set, then "readWrite" is automatically set.
• "randomAccess" is set to true for binary files.
• "autoflush" defaults to true

Valid types
"text"	open the file for text access, using the default file encoding.
"binary"	open the file for binary access.
"unicode"	open the file for unicode access.

Causes a warning if the file is a native file or if the file is a directory. In both these cases the return value is true. If the file is non-native and already open, a warning is generated and the file is reopened. This might make sense in a case when you want to go to the beginning of the file,etc., but you can probably use seek  instead.

close()

This method closes the file. Causes a warning if the file is not open or is a native file and returns false. This method is called automatically on an open file object when the object goes out of scope.

remove()

This method removes the file or removes the directory given as argument. Normally if remove is called on a directory it is only removed if the directory is empty. Fails if the file is open.

copyTo(filename)

This method tries to make a raw copy of the source file into a destination file. It will fail if the source file doesn't exist, or is a directory.

renameTo(filename)

This method moves/renames the file. Fails if the file is open.

flush()

This method forces to flush the output buffers of the file. Fails if the file is closed.

seek(numChars)

This methods skips the next numChars characters or bytes depending on the current access mode for the file. If the file is closed, it is opened before performing the operation.This function can accept both positive and negative parameters. Reports a warning and returns undefined if called on a directory. In the operation is successful, returns the current position in file.

read(numChars)

This method reads numChars characters and returns them in a string. If there are less than numChars characters left to read, then the available characters are returned. If the file is at EOF, then null is returned.

readln()

This methods reads characters until an end of line/file is encountered then returns the string for these characters (without any end of line character).

readAll()

This methods reads the rest of the file and returns an array with a slot for each line of the file.

write(arg0,arg1,...,argn)

This method converts each argument to a string, then writes it to the file (without separators).

writeln(arg0,arg1,...,argn)

This method is similar to write, but adds a platform dependent end of line after outputting the last argument.

writeAll(Array)

This method takes an array as an argument, and calls writeln() on each element.

list(filter)

This method returns an array. if the file is a directory, an slot is created for each file inside that directory, and a property with the file name is created too. Both point to a File object referring to the file.

filter is an optional argument that should be a function or a regular expression, and is used to filter the array. If the argument is a regular expression, the array will only contain files for which the pattern matches. If the argument is a function, the array will only contain files for which the function returned true when called with the file name as argument.
 

Example

File.currentDir.list( function (name) { return name.length==3; } ); Returns only files in the current directory whose names are 3 characters long.

mkdir()

This method attempts to create a directory inside the file directory. For instance,

f = new File("c:\\")
f.mkdir("win95")
creates c:\win95\ and this piece does the same
f = new File("c:\\user.txt")
f.mkdir("win95")
toString()

This method returns the canonical path to the file.

toURL()

return the name of the file as a "file:" URL. Undefined for pipes and standard streams.

4.0 I want to play

Flaming disclaimer: Do not play with the File object if you are not prepared to have your hard drive erased, smashed, and broken into little bits! It mostly works right now, but no guarantees.

 
Currently the File object can be built into the core JS engine using a define. This is wrong. Once XPCOM settles down, the File object should be COM-ified.

To play with/hack on the File object, get the latest and greatest (usually) JavaScript using CVS:

cvs co -rSpiderMonkeyDev_BRANCH mozilla/js/src

Be warned that this gives you the active development branch for JavaScript, which offers few guarantees of stability.

To build (the reference interpreter on UNIX in this example) with the File object, define JS_HAS_FILE_OBJECT while building. You also need to build against NSPR which you can do by defining JS_THREADSAFE.

cd mozilla/js/src
gmake -f Makefile.ref JS_THREADSAFE=1 JS_HAS_FILE_OBJECT=1

5.0 Status

Status of the code:
• All code is in jsfile.[c|h|msg]. It is in a pretty good shape, it any case, in a significantly better shape than it was in mid-December, 1998. There are still some opportunities for removing redundant pieces, adding macros, etc.
• Generally, most important functionality is implemented and tested. This includes opening, closing files, reading and writing, etc. See nspr_files.js in the test suite for more examples.
• I believe all methods and properties described in this proposal have been implemented. This doesn't necessarily mean that they work on all platforms. In paraticular, no one has tried to build this thing on the Mac. Note that some functionality such as pipes and perhaps even standard streams is not going to be available on the Mac. Perhaps not even all NSPR functions will work. It's not even clear which Mac includes are needed.
• Speaking about the code, most methods of the File object are pretty well tested. There may still be random bugs in the code dealing with low-level file access. That code is also not optimized, copying a relatively big file takes a while. Filename manipulation has been rewritten. There has been a talk about combining this stuff with what John McMullen has written and that shouldn't be terribly hard, once they (NSPR people, Brendan) figure out it it should be part of NSPR and which implementation language to use. There is a discussion about this on netscape.public.mozilla.xpfe.
• There is a test suite covering most of the functionality. However, I expect there will be many subtle bugs when the object is heavily used. They are usually hard to discover, but easy to fix.
• A lot of the code is dedicated to catching error conditions and generating error messages or warnings. See jsfile.msg for error codes.

Things to do:
• Exceptions should be generated when errors occur. What should the format of these be? Some of the functions report warnings, some errors. Ultimately, all of them should eventually become exceptions.
• Error messages should be cleaned up a bit
• There has been a talk about putting security callbacks into the code. There is a SECURITY_CHECK macro left for this purpose. It is currently defined as empty string and needs to be redefined. This macro takes the name of the operation as one of the parameters, these names currently correspond to method names, but can probably be broken into groups (read, write, exist, etc.). Security checks are implemented at the level of individual methods and property access procedures, not at the level of underlying implementation functions because  implementation functions can be called in the code from different places and we probably don't want the security checks to be layered. Ask Norris about this.
• RDF will be reflected into JS (Ask Guha, Clayton). Currently, we'd like to have a JavaScript file constructor taking an RDF thingy and returning a normal JS File object. That is, one would use RDF to naviate the file system and the File object to do reads, writes, etc.
• Figure out what to do with js_canonicalPath(...). This procedure resolves a filename give something like ".././blah/../../dir/file.txt". I think, it is not really needed, since the filesystem will resolve this name automatically (it all boils out to an fopen call at the end). The procedure is not Mac-compatible and there may potentially be weird cases and soft link problems. In the code, look for macro RESOLVE_PATH. This procedure also does some other things such as removing leading and trailing spaces, they may need to be moved to js_absolutePath(...).
• It would be nice to run the code through Purify.
• js_escape from jsstr.h is used in the code. Some day it may disappear from jsstr.h and will be moved to the client. And the code won't build...
• (?) Get NSPR group to roll nsFileSpec functionality into NSPR. See discussion above. It's not completely clear if this is what we want to do since out code is already more or less working, but it is worth looking into.
• (?) XPCOM-ify -- again, not clear what this gives us, but this will requre a major rewrite.
• (?) Implement in Java, not clear why, LiveConnect + java.io.File already give us this
• (?) Extend api to allow use of sockets, etc. (functionality already exists in PRFile). It is not clear why this it needed, but if it's easy to do, why not.
• (?) Low-level file-access functions are probably not efficient (see js_FileRead(..), etc.) Copying a relatively small file takes a while. They can probably be optimized, though there is already some buffering in place.


Random thoughts:

• Someone in the server land has tried to write something similar (Server-side JS?) and there also seems to be an XPCOM file object somewhere. (Ask Clayton).