Header Mastering Node js Part 3 Node js

Header Mastering Node. js, Part 3 Node. js Files, Stream, and Buffers Microsoft Virtual Academy Eric W. Greene Produced by

Course Overview Getting Started with Files, Streams and Buffers Course Project – Simple Web Server Reading and Writing to/from Files Understanding Buffers File System Tasks Using Streams Conclusion

Course Prerequisites The course assumes: Good Knowledge of Java. Script Good Knowledge of Node. js and Common. JS Modules To run the examples on your machine, you will need Node. js installed Node. js can be downloaded from here: https: //nodejs. org/en/ Please install version 6 or later If you need a refresher on these technologies, then watch the Wintellect. NOW courses, Introduction to Node. js and Node. js Modules

Files, Streams and Buffers Node. js provides several APIs for working with files, streams and buffers Files, streams and buffers are implemented in C++, and made accessible to an application's Java. Script code through the Node. js API Node. js is a collection of C++ modules accessible via the V 8 Java. Script Runtime Java. Script does not directly utilize resources such as files, the Node. js C++ modules perform requests on behalf of an application's Java. Script code through an API provided by Node. js

Course Project To provide practical context to the course material a very simple web server project will be implemented using the Node. js Http module The web server will coded to perform the following operations: Handle requests for files from the file system Log interactions with the web server to a file on the file system Use buffers to receive HTTP POST data via a stream Rotate log files with file system operations Compress HTTP responses with streams and the zlib library

Node. js Modules for Working with Files There are two primary modules used to work with files and the file system Path – File/Folder Paths FS – File System The path module is used to create paths for working with files and folders. Because Node. js runs on multiple platforms, the path module creates path based upon operating system differences The fs module supports the creation, modification, and deletion of files/folders

Setting up the Course Project

Working with Paths Supports POSIX and Windows paths Commonly used functions: dirname – returns the folder portion of the path basename – returns the file name portion of the path extname – returns the extension portion of the path format – returns a properly formatted path for the operating system

Working with Paths Commonly used functions (continued): join – combines path segments using the operating system path separator normalize – simplifies a path by resolving '. ' and '. . ' segments parse – returns object with the component parts of a path

Exploring the Path Module

Reading a File Node. js supports reading text and binary files Files can be read in chunks or all at once The open and read function can be used in conjunction to read a file in small chunks The read. File function can be used to read an entire file at once Both the read and read. File functions are asynchronous, and have analogous synchronous implementations named read. Sync and read. File. Sync

Reading a File Chunk by Chunk The open function is used to open a file, and returns a file descriptor The mode "r" signifies the file is being opened for reading, there are many modes Once opened, the file descriptor is passed to the read function, where the a buffer is populated with data from the file The data to be read is specified by a length, and an optional starting position If no starting position is specified, then the current file position is used

Reading a File Chunk by chunk is most useful when only portions of the file need to be read The file must be closed with the close function

Reading an Entire File For most files, reading the entire file is much simpler than reading chunk by chunk When reading the whole file, the file encoding should be specified the data returned will be match the desired result Does not require opening the file first, a string file name can be specified If a file descriptor is passed, read. File does not close the file

Reading an Entire File

Writing to a File Writing to a file is very similar to reading from a file, except the content to be written must be specified Both binary and text data can be written to a file The functions open and write can be used to write chunk by chunk, or the function write. File can be used to write an entire file at once Files can be overwritten and appended to There asynchronous and synchronous versions of write and write. File

Writing a File Chunk by Chunk The open function is used to open a file, and returns a file descriptor The mode "w" signifies the file is being opened for writing, there are many other modes such as read/write and appending If the file does not exist then it is created, if the file does exist then its overwritten Once opened, the file descriptor is passed to the write function A String or Buffer is passed and written at the specified position, if no position if specified, then the data is written at the current position

Writing an Entire File For most files, writing the entire file is much simpler than writing chunk by chunk The function write. File will write an entire String or Buffer to a file The encoding of the data should be specified Does not require opening the file first, a string file name can be specified If a file descriptor is passed, write. File does not close the file

Appending to a File In addition to reading a file, and writing to a file, its possible to append to an existing file There are two function for appending to files, append. File and append. File. Sync These two files follow the same pattern as write. File and read. File Additionally, when opening a file with the open function, the mode "a" can be used to append to a file

Append to a File

Asynchronous or Synchronous? As a general rule, all resource interactions including file access should be asynchronous Synchronous file interactions block Java. Script from executing additional tasks while file system interaction is occurring Synchronous file access can be useful for reading configuration files when a program is starting up, but once the application is interacting with the user, all file system interactions should be asynchronous

File Existence & Getting File Information Checking for file existence does not make sense in Node. js Checking for file existence before opening a file introduces the possibility of a race condition as the status of the file could change between the time it is checked, and the time it is opened If the code simply needs to check for existence, then the stat or access function should be used

File Existence & Getting File Information If the code is merely checking to verify the file is there before opening it, then the code should attempt to open it, and handle the error if the file is not there - this is best practice for handling situations where the file does not exist The exists and exists. Sync function are still present in Node. js 6, but they have been deprecated for many pass major versions, and should not be used

Performing File System Tasks In addition to reading and writing files, the fs module can perform many file system operations Folders and temporary folders can be created and destroy, and folder contents can be read Files and folders can be watched for changes, and when changes occur, code can be executed to handle those changes Many of the Unix files operations such as link and chmod can be accomplished if the operating system supports those operations

Performing File System Tasks Commonly used functions: mkdir – creates a folder rmdir – deletes a folder rename – renames files and folders readdir – returns the contents of a folder

Performing File System Tasks Commonly used functions (continued): watch – watches a file or folder for changes, uses operating system specific functions watch. File – better to use watch but will work in situations where watch does not, its uses polling which is inefficient

Perform File System Operations

Buffers hold binary data The are created with a fixed size, but can be concatenated or sliced to produce new buffers Buffers are commonly used to hold data when completing IO operations such as reading data from a file While a buffer size is fixed, the contents of the buffer can be modified permitting binary data manipulations

Streams Node. js supports the streaming of data Think of a stream of water, the water continuously flows from a source to its mouth Streams of data are like a stream of water, data flows from some kind of source, and code can be written to receive the flowing data Like a river dam, streams can be corked/paused to stop the flow of data, and uncorked/resumed to continue the data Streams typically stream String and Buffer objects. Streams can run in object mode serving up any kind of data except for null

Streams Node. js provides the stream module to allow developers to create new object which implements the stream interface and streams data The focus on the course is not the creation of new streams; but rather, how to consume existing streams There are many example of streams in Node. js such as HTTP requests and responses, and reading and writing files to name a few All of the streams for Node. js APIs only serve String and Buffer objects, they are not object mode streams

Reading from a Stream To read data from a stream the consumer can listen for the data event When data is ready to be consumed, the event is fired, and a callback function is invoked with the data When no more data is available, the end event is fired In addition to listening the data event, a readable stream can pipe its data into a writable stream The data can be a String or a Buffer

Reading from a Stream

Writing to a Stream Writing to stream requires a writable stream such as a file writer stream or a zlib compression stream Streams can be written to directly through the write function Readable streams can be piped into writable streams through the pipe function Use the finish event on the writable stream to know when writing has completed

Writing to a Stream

Conclusion Using Node. js to work with files, folders and streams of data is very useful Java. Script itself does not actually interact with the file system; instead, Node. js provides APIs which Java. Script uses to interact with C++ modules to do those operations Handing off IO operations to C++ modules allows Java. Script to process more Java. Script tasks waiting for the C++ modules to complete operations and pass back the data via callback functions – this is called Asynchronous programming Node. js supports useful features such as Streams and Buffers