V8 Fatal Error when trying to convert a buffered integer to string

IO.js Version 1.0.4

$ iojs
> Date.now();
1422508723712
> Buffer(Date.now());
<Buffer 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ... >
> Buffer(Date.now()).toString('utf8');


#
# Fatal error in ../deps/v8/src/handles.h, line 48
# CHECK(location_ != NULL) failed
#

==== C stack trace ===============================

 1: V8_Fatal
 2: v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::String::NewStringType, int)
 3: node::StringBytes::Encode(v8::Isolate*, char const*, unsigned long, node::encoding)
 4: node::Buffer::Utf8Slice(v8::FunctionCallbackInfo<v8::Value> const&)
 5: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&))
 6: ??
 7: ??
Illegal instruction

Date.now() is an integer, so you're creating a buffer with Date.now() bytes... which is gigantic. I think you want new Buffer(String(Date.now()))

I understand what's wrong, haha. I'm just wondering why its giving a scary V8 error like that instead of a generic Javascript error.

In Node 0.10.31:

$ node
> Buffer(5).toString('utf8')
'\u0000 \u0000\u0000\u0000'
> Buffer(Date.now()).toString('utf8')
TypeError: Bad argument
    at new Buffer (buffer.js:194:21)
    at Buffer (buffer.js:158:12)
    at repl:1:2
    at REPLServer.self.eval (repl.js:110:21)
    at Interface.<anonymous> (repl.js:239:12)
    at Interface.emit (events.js:95:17)
    at Interface._onLine (readline.js:202:10)
    at Interface._line (readline.js:531:8)
    at Interface._ttyWrite (readline.js:760:14)
    at ReadStream.onkeypress (readline.js:99:10)

@kevinmartin

@kevinmartin well, it does have an extraordinary length.

> var buf = new Buffer(Date.now())
undefined
> buf.length
878611666

this issue reminds me of similar crash I had last year on node. I just retried it on iojs and it is still crashing.

test.js :

var fs = require('fs');
var zlib = require('zlib');

fs.readFile('./Compound_006625001_006650000.sdf.gz', function (err, content) {
  zlib.gunzip(content, function (err, sdf) {
    sdf.toString();
  });
});

The gzipped file : ftp://ftp.ncbi.nlm.nih.gov/pubchem/Compound/CURRENT-Full/SDF/Compound_006625001_006650000.sdf.gz

> iojs test.js
#
# Fatal error in ../deps/v8/src/handles.h, line 48
# CHECK(location_ != NULL) failed
#

==== C stack trace ===============================

 1: V8_Fatal
 2: v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::String::NewStringType, int)
 3: node::StringBytes::Encode(v8::Isolate*, char const*, unsigned long, node::encoding)
 4: node::Buffer::Utf8Slice(v8::FunctionCallbackInfo<v8::Value> const&)
 5: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&))
 6: ??
 7: ??
zsh: illegal hardware instruction (core dumped)  iojs test.js

The uncompressed file is valid utf-8.
Could it be a v8 limit on the size (it is 276,9 MB) ?

There is a logic error in lib/buffer.js: the length argument to the constructor gets coerced to uint32 with length >>> 0, which makes it bypass the length < kMaxLength check because e.g. (1422546646261 >>> 0) < 0x3fffffff.

In short, confirmed. :-)

@bnoordhuis

I think @bnoordhuis found a bug, but I don't think it was this bug. The length > kMaxLength after length >>> 0 check he mentions is part of SlowBuffer(), and that's not called at all in this stack trace.

We just have a plain Buffer (for which util.isNumber(subject) returns true), and when it hits the if (this.length > kMaxLength) check at lib/buffer.js:52 it has a length of 921032252 and a kMaxLength of 1073741823, so it's under the limit at 85% of kMaxLength.

See #657 for a proposed mitigation. It's not perfect because the real issue lies in V8, it aborts when internal operations OOM. For example, Buffer(0x3fffffff-1).toString() still hits the assert.

V8 bug report: https://code.google.com/p/v8/issues/detail?id=3853

Also, the cutoff point that's currently in there was chosen after meticulous benchmarking. Here's a basic graph showing what happens to allocation time as string size increases: http://trevnorris.github.io/nodeconf.eu/#/17

External strings can only be one or two byte strings. Thus excluding the possibility of an external UTF8 string.

True, but internal strings aren't UTF-8 either, V8 always decodes them to ASCII or UTF-16.

True, but internal strings aren't UTF-8 either, V8 always decodes them to ASCII or UTF-16.

Yes, we are not gaining anything by outsourcing this work to v8. We can do it ourselves and make it external.

Sure, but do we really want to implement our own encoding parser so we can externalize all strings?

We don't have to implement our own, even though it's a fun task.

test: fix test for buffer regression #649

test: fix test for buffer regression nodejs#649

test: fix test for buffer regression #649

Navigation Menu

V8 Fatal Error when trying to convert a buffered integer to string #649

11 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

V8 Fatal Error when trying to convert a buffered integer to string #649

Description

Activity

jonathanong commented on Jan 29, 2015

kevinmartin commented on Jan 29, 2015

brendanashworth commented on Jan 29, 2015

targos commented on Jan 29, 2015

bnoordhuis commented on Jan 29, 2015

cjb commented on Jan 29, 2015

bnoordhuis commented on Jan 29, 2015

bnoordhuis commented on Jan 29, 2015

11 remaining items

trevnorris commented on Mar 13, 2015

bnoordhuis commented on Mar 13, 2015

petkaantonov commented on Mar 13, 2015

trevnorris commented on Mar 13, 2015

petkaantonov commented on Mar 13, 2015

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions