Illegal characters in JSON response - 【StackMirror】|javascript|json|unicode|utf-8

I have a Sencha Touch app. One of the stores I have uses an ajax proxy and a json reader. Some of the strings in the JSON returned from my sinatra app occasionally contain this character: http://www.fileformat.info/info/unicode/char/2028/index.htm

Although it's invisible, the character occurs twice in the second string here, between the period and the ending quote:

"description": "Each of the levels requires logic, skill, and brute force to crush the enemy.  "

Try copy and pasting "Each of the levels requires logic, skill, and brute force to crush the enemy.  " into your javascript console! It won't be parsed as a string, and fails with SyntaxError: Unexpected token ILLEGAL.

This causes the JSON response to fail. I've been stuck on this for a long time! Any suggestions?

2012-04-04 22:35
by nnyby

Why not just remove the bad character from output JSON string on server side - Marat Tanalin 2012-04-04 22:50

@MaratTanalin I'm looking into that now, but until recently I didn't have access to the server. I guess there is no client-side solution - nnyby 2012-04-04 22:52

If the string is illegal JavaScript literal (not sure about your case, but, for example, unescaped line feeds are not allowed in JS literals), then client-side solution unlikely exist. Though maybe you could try to get JSON Ajax response as text instead of JSON, then filter undesired character from it as string, and then parse filtered string as JSON string. Unfortunately it's a workaround. It would be better to find specific nature of the illegal character in your JSON strings and resolve the reason - Marat Tanalin 2012-04-04 23:03

Sure there is. You just run some .replace() filters on the string before you eval() it - Travis Webb 2012-04-05 00:34

@TravisWebb is right. Just filter out the string before passing it to the JSON decode function - Chris Laplante 2012-04-05 02:11

You should just make sure that your server is generating valid JSON. What are you using to generate the JSON? in PHP, if you pass it a string with an embedded newline character to json_encode, it gets converted to \r\ - Juan Mendes 2012-04-05 21:09

Like you're been told, either escape the control character (preferably already server side, maybe you'll be hooking on this JSON with other apps), or replace it from the string before parsing JSON from the server response. And @TravisWebb, don't use eval() : - Zlatko 2012-04-06 11:48

possible duplicate of Javascript parse error on '\u2028' unicode characterbernie 2012-04-07 00:23

Thanks @bernie this question is in fact a duplicate of that one. There is some good info there. If anyone is wondering I ended up solving this on the server: http://stackoverflow.com/questions/10020962/strip-ruby-string-of-a-specific-control-characte - nnyby 2012-04-08 21:01

The only reliable way to fix this is server-side. Make sure your JSON generator emits those characters escaped, e.g. as \u2028.

In my experience, it's easiest to simply encode your JSON in plain ASCII which will always work. The downside is that it's less efficient as non-ASCII characters will take up more space, so depending on the frequency of those, you may not want that trade-off...

The documentation for Perl's JSON::XS has a good explanation of the problem and advice for how to fix it in Perl: http://search.cpan.org/perldoc?JSON::XS#JSON_and_ECMAscript

2012-05-03 01:15
by Lasse

Link fixed. Thanks - Lasse 2014-02-24 12:11

Conceptually you are only allowed to send out strings from the server that are valid JavaScript literals by escaping appropriately.

If you want to fix this issue on the client you need an extra workaround step (only seems to work in Firefox):

var a = escape("Each of the levels requires logic, skill, and brute force to crush the enemy.");
alert(unescape(a));

But the discussion is obsolete, because you must escape on the server.

2012-04-16 11:35
by Sam

The code won't parse so you cannot fix it on the client side - Salman A 2012-04-16 11:38

For me it worked on ff with firebug, have you even tried it - Sam 2012-04-16 18:12

I tried it in Chrome. Copied the portion of OP's code after : and pasted it in console, got syntax error - Salman A 2012-04-17 05:36

-2

Avoid using eval to parse JSON.

Use JSON.parse or https://github.com/douglascrockford/JSON-js.

JSON.parse('{}');  // where {} is your JSON String

2012-05-03 02:30
by Hello71