Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
"readme":"[](http://travis-ci.org/soldair/node-watchfd)\n\n## watchfd\n\nWatch events open,change,unlink on all files that are refrenced or become refrenced by path\n\nprovide events for any file descriptors that are referenced by a watched path, \nor were referenced by a watched path for as long as they are still changing.\nactive is defined by a timeout since last event. file descriptors that become inactive are removed.\n\n\n## install\n\n\tnpm install watchfd\n\n## use\n\n\tvar watchfd = require('watchfd').watch;\n\twatchfd('/some.log',function(cur,prev){\n\t\tconsole.log(prev.size,' changed to ',cur.size);\n\t});\n\n### a use case\n\nan issue with log/file forwarding utilities currently available in npm is that they only watch the file descriptor under the filename. when a log is rotated and a new log is created the server may not stop writing to the old file descriptor immediately. Any data written to that descriptor in this state ends up in /dev/null\n\n\n### argument structure\n\nwatchfd.watch(filename, [options], listener)\n\n- filename\n its really intended that this be a regular file or non existant. i dont know what would happen right now if its a directory.\n- options. supported custom options are\n\n\t```js\n\t{\n\t\"timeout\": 60*60*1000, //defaults to one hour\n\t//how long an inactive file descriptor can remain inactive\n\n\t\"timeoutInterval\":60*5*1000 //every five minutes\n\t// how often to check for inactive file descriptors\n\t}\n\n\t//the options object is also passed directly to watch and watchFile so you may configure\n\n\t{\n\t\"persistent\":true, //defaults to true\n\t//persistent indicates whether the process should continue to run as long as files are being watched\n\n\t\"interval\":0, //defaults 0\n\t//interval indicates how often the target should be polled, in milliseconds. (On Linux systems with inotify, interval is ignored.) \n\t}\n\t```\n\n- callback\n this is bound to the change event of the watcher. its required\n\n\t```js\n\tcallback(cur,prev)\n\t```\n\n cur and prev are instances of fs.Stats\n\n- @returns\n an instance of Watcher\n\n### Watcher methods\n\nWatcher.pause()\n\n- paused, changed and last state is kept for each file descriptor\n - stops file descriptors from timing out.\n - all events except error are paused.\n - unlink, open, change etc will be fired in the correct order after resume. \n no events will be missed but change events will be combined\n\n\nWatcher.resume()\n\n- resumed\n - for each file descriptor pending events are fired in the corect order\n open,change,unlink\n - the change event has the stat from first change event while paused and the most recent so no change is missed.\n\n\nWatcher.paused\n\n - is paused\n - readonly please.\n\n### Watcher events\n\nWatcher.on(event name,call back);\n\n- change\n\t\tfs.Stats cur, fs.Stats prev\n- open\n\t\tfs.Stats cur,{fd:file descriptor,stat:fs.Stats cur}\n- unlink\n fs.Stats cur,{fd:file descriptor,stat:fs.Stats cur}\n- timeout\n fs.Stats cur,{fd:file descriptor,stat:fs.Stats cur}\n\n#### windows support problems\n\n- It uses file inode as a unique id for each descriptor. I know there is a way to get a unique id for a file in windows i just don't know if that would be passed to stat as stat.ino. \n- I use watchFile which is not supported at all on windows but this would be easier to overcome considering i can use a configured polling interval as a stat polling fall back on windows. \n- I also don't know windows very well and don't know if windows has the problem this module solves...but i imagine it would\n\n#### notes\n\nI noticed distinct differences in watchFile vs watch api\nfs.watchFile will issue events for a file that is currently referenced by a path\nfs.watch will take a path but issue events whenever that file descriptor is changed even after it's unlinked\n\nWe should probably design servers to listen to SIGHUP and grab new file descriptors for all loggers but even if you used logrotate with copytruncate mode as to not change the file referenced by a path the chance that you will loose data is still there. I feel safer waiting for a file descriptor to be quiet so i know its out of use before i close it in a process that has the ability to read data out of it.\n",
Watch events open,change,unlink on all files that are refrenced or become refrenced by path
provide events for any file descriptors that are referenced by a watched path,
or were referenced by a watched path for as long as they are still changing.
active is defined by a timeout since last event. file descriptors that become inactive are removed.
## install
npm install watchfd
## use
var watchfd = require('watchfd').watch;
watchfd('/some.log',function(cur,prev){
console.log(prev.size,' changed to ',cur.size);
});
### a use case
an issue with log/file forwarding utilities currently available in npm is that they only watch the file descriptor under the filename. when a log is rotated and a new log is created the server may not stop writing to the old file descriptor immediately. Any data written to that descriptor in this state ends up in /dev/null
### argument structure
watchfd.watch(filename, [options], listener)
- filename
its really intended that this be a regular file or non existant. i dont know what would happen right now if its a directory.
- options. supported custom options are
```js
{
"timeout": 60*60*1000, //defaults to one hour
//how long an inactive file descriptor can remain inactive
"timeoutInterval":60*5*1000 //every five minutes
// how often to check for inactive file descriptors
}
//the options object is also passed directly to watch and watchFile so you may configure
{
"persistent":true, //defaults to true
//persistent indicates whether the process should continue to run as long as files are being watched
"interval":0, //defaults 0
//interval indicates how often the target should be polled, in milliseconds. (On Linux systems with inotify, interval is ignored.)
}
```
- callback
this is bound to the change event of the watcher. its required
```js
callback(cur,prev)
```
cur and prev are instances of fs.Stats
- @returns
an instance of Watcher
### Watcher methods
Watcher.pause()
- paused, changed and last state is kept for each file descriptor
- stops file descriptors from timing out.
- all events except error are paused.
- unlink, open, change etc will be fired in the correct order after resume.
no events will be missed but change events will be combined
Watcher.resume()
- resumed
- for each file descriptor pending events are fired in the corect order
open,change,unlink
- the change event has the stat from first change event while paused and the most recent so no change is missed.
- It uses file inode as a unique id for each descriptor. I know there is a way to get a unique id for a file in windows i just don't know if that would be passed to stat as stat.ino.
- I use watchFile which is not supported at all on windows but this would be easier to overcome considering i can use a configured polling interval as a stat polling fall back on windows.
- I also don't know windows very well and don't know if windows has the problem this module solves...but i imagine it would
#### notes
I noticed distinct differences in watchFile vs watch api
fs.watchFile will issue events for a file that is currently referenced by a path
fs.watch will take a path but issue events whenever that file descriptor is changed even after it's unlinked
We should probably design servers to listen to SIGHUP and grab new file descriptors for all loggers but even if you used logrotate with copytruncate mode as to not change the file referenced by a path the chance that you will loose data is still there. I feel safer waiting for a file descriptor to be quiet so i know its out of use before i close it in a process that has the ability to read data out of it.
"description":"Tail a file. This will continue to work even if a file is unlinked rotated or truncated. It is also ok if the path doesnt exist before watching it",
"readme":"[](http://travis-ci.org/soldair/node-tailfd)\n\n## tailfd\n\nTails a file. it should work great. it will continue to work even if a file is unlinked rotated or truncated. It is also ok if the path doesnt exist before watching it\n\n## Example\n\n```js\n\nvar tail = require('tailfd').tail,\nwatcher = tail('/some.log',function(line,tailInfo){\n //default line listener. optional.\n console.log('line of data> ',line);\n});\n\n```\n\nwhen you are done\n\n```js\nwatcher.close();\n\n```\n\n## install\n\n\tnpm install tailfd\n\n### argument structure\n\ntailfd.tail(filename, [options], listener)\n\n- filename\n - this should be a regular file or non existent. the behavior is undefined in the case of a directory.\n\n- options. supported custom options are\n\n\t```js\n\t{\n\n\t\"start\":undefined, //defaults to the first reported stat.size\n\t//optional. a hard start position in the file for tail to start emitting data events.\n\n\t\"offset\":0,\n\t//optional. offset is negtively applied to the start position\n\n\t\"delimiter\":\"\\n\",\n\t//optional. defaults to newline but can be anything\n\n\t\"maxBytesPerRead\":10240,\n\t// optional. this is how much data will be read off of a file descriptor in one call to fs.read. defaults to 10k.\n\t// the maximum data buffer size for each tail is \n\t// \tmaxBufferPerRead + the last incomplete line from the previous read.length\n\n\t\"readAttempts\":3,\n\t//optional. if fs.read cannot read the offset from a file it will try attempts times before is gives up with a range-unreadable event\n\t// defaults to 3 attempts\n\n\t\"maxLineLength\":102400\n\t// optional. defaults to 1 mb\n\t// if the line excedes this length it's data will be emitted as a line-part event\n // this is a failsafe so that a single line cant eat all of the memory.\n\t// all gathered line-parts are complete with the value of the next line event for that file descriptor.\n\n\t}\n\n\t// the options object is passed to watchfd as well. With watchfd you may configure\n\n\t{\n\n\t\"timeout\": 60*60*1000, //defaults to one hour\n\t//how long an inactive file descriptor can remain inactive before being cleared\n\n\t\"timeoutInterval\":60*5*1000 //every five minutes\n\t// how often to check for inactive file descriptors\n\n\t}\n\n\t//the options object is also passed directly to fs.watch and fs.watchFile so you may configure\n\n\t{\n\t\"persistent\":true, //defaults to true\n\t//persistent indicates whether the process should continue to run as long as files are being watched\n\n\t\"interval\":0, //defaults 0\n\t//interval indicates how often the target should be polled, in milliseconds. (On Linux systems with inotify, interval is ignored.) \n\t}\n\n\t```\n\n- callback\n - this is bound to the line event of the watcher. its optional.\n\n\t```js\n\tcallback(line,tailInfo)\n\n\t```\n\n cur and prev are instances of fs.Stats\n\n- @returns\n TailFD Watcher\n\nWatcher.pause\n- pause data and line events on all underlying descriptors\n\nWatcher.resume\n- get it goin again! =)\n\n\n### events\n\n- line\n\t- String line, Object tailInfo\n- data\n\t- Buffer buffer, Object tailInfo\n- line-part\n\t- String linePart, Object tailInfo\n\t\t- if line length excedes the options.maxLineLength the linePart is emitted. \n\t\t This is to prevent cases where unexpected values in logs can eat all of the memory.\n- range-unreadable\n\t- Array errors, Number fromPos,Number toPos,Object tailInfo\n\t\t- After configured readAttempts the offset could still not be read. This range will be skipped\n\n### events inherited from watchfd\n\n- change\n\t- fs.Stats cur, fs.Stats prev\n- open\n\t- fs.Stats cur,{fd:file descriptor,stat:fs.Stats cur}\n- unlink\n\t- fs.Stats cur,{fd:file descriptor,stat:fs.Stats cur}\n- timeout\n\t- fs.Stats cur,{fd:file descriptor,stat:fs.Stats cur}\n\n### tailInfo properties\n\n- stat\n\t- instanceof fs.Stats\n- pos\n\t- current seek position in the file\n- fd\n\t- file descriptor being tailed\n- buf\n\t- string containing the last data fragment from delimiter parsing\n\n\n#### watch file and watch may behave differently on different systems here is the doc for it.\n\n- http://nodejs.org/api/fs.html#fs_fs_writefile_filename_data_encoding_callback\n- http://nodejs.org/api/fs.html#fs_fs_watch_filename_options_listener\n",
Tails a file. it should work great. it will continue to work even if a file is unlinked rotated or truncated. It is also ok if the path doesnt exist before watching it
## Example
```js
vartail=require('tailfd').tail,
watcher=tail('/some.log',function(line,tailInfo){
//default line listener. optional.
console.log('line of data> ',line);
});
```
when you are done
```js
watcher.close();
```
## install
npm install tailfd
### argument structure
tailfd.tail(filename, [options], listener)
- filename
- this should be a regular file or non existent. the behavior is undefined in the case of a directory.
- options. supported custom options are
```js
{
"start":undefined, //defaults to the first reported stat.size
//optional. a hard start position in the file for tail to start emitting data events.
"offset":0,
//optional. offset is negtively applied to the start position
"delimiter":"\n",
//optional. defaults to newline but can be anything
"maxBytesPerRead":10240,
// optional. this is how much data will be read off of a file descriptor in one call to fs.read. defaults to 10k.
// the maximum data buffer size for each tail is
// maxBufferPerRead + the last incomplete line from the previous read.length
"readAttempts":3,
//optional. if fs.read cannot read the offset from a file it will try attempts times before is gives up with a range-unreadable event
// defaults to 3 attempts
"maxLineLength":102400
// optional. defaults to 1 mb
// if the line excedes this length it's data will be emitted as a line-part event
// this is a failsafe so that a single line cant eat all of the memory.
// all gathered line-parts are complete with the value of the next line event for that file descriptor.
}
// the options object is passed to watchfd as well. With watchfd you may configure
{
"timeout": 60*60*1000, //defaults to one hour
//how long an inactive file descriptor can remain inactive before being cleared
"timeoutInterval":60*5*1000 //every five minutes
// how often to check for inactive file descriptors
}
//the options object is also passed directly to fs.watch and fs.watchFile so you may configure
{
"persistent":true, //defaults to true
//persistent indicates whether the process should continue to run as long as files are being watched
"interval":0, //defaults 0
//interval indicates how often the target should be polled, in milliseconds. (On Linux systems with inotify, interval is ignored.)
}
```
- callback
- this is bound to the line event of the watcher. its optional.
```js
callback(line,tailInfo)
```
cur and prev are instances of fs.Stats
- @returns
TailFD Watcher
Watcher.pause
- pause data and line events on all underlying descriptors
Watcher.resume
- get it goin again! =)
### events
- line
- String line, Object tailInfo
- data
- Buffer buffer, Object tailInfo
- line-part
- String linePart, Object tailInfo
- if line length excedes the options.maxLineLength the linePart is emitted.
This is to prevent cases where unexpected values in logs can eat all of the memory.
- range-unreadable
- Array errors, Number fromPos,Number toPos,Object tailInfo
- After configured readAttempts the offset could still not be read. This range will be skipped
