How to convert UTF-8 byte[] to string?

Go To StackoverFlow.com

814

I have a byte[] array that is loaded from a file that I happen to known contains UTF-8. In some debugging code, I need to convert it to a string. Is there a one liner that will do this?

Under the covers it should be just an allocation and a memcopy, so even if it is not implemented, it should be possible.

2009-06-16 18:47
by BCS
"should be just an allocation and a memcopy": is not correct because a .NET string is UTF-16 encoded. A Unicode character might be one UTF-8 code unit or one UTF-16 code unit. another might be two UTF-8 code units or one UTF-16 code unit, another might be three UTF-8 code units or one UTF-16 code unit, another might be four UTF-8 code units or two UTF-16 code units. A memcopy might be able to widen but it wouldn't be able to handle UTF-8 to UTF-16 conversion - Tom Blodget 2016-11-19 01:01


1299

string result = System.Text.Encoding.UTF8.GetString(byteArray);
2009-06-16 18:49
by Zanoni
how does it handle null ended strings - maazza 2015-05-12 12:43
@maazza for unknown reason it doesn't at all. I'm calling it like System.Text.Encoding.UTF8.GetString(buf).TrimEnd('\0'); - Hi-Angel 2015-07-27 07:53
@Hi-Angel Unknown reason? The only reason null-terminated strings ever became popular was the C language - and even that was only because of a historical oddity (CPU instructions that dealt with null-terminated strings). .NET only uses null-terminated strings when interopping with code that uses null-terminated strings (which are finally disappearing). It's perfectly valid for a string to contain NUL characters. And of course, while null-terminated strings are dead simple in ASCII (just build until you get the first zero byte), other encodings, including UTF-8, are not so simple - Luaan 2015-11-23 10:05
One of the beautiful features of UTF-8 is that a shorter sequence is never a subsequence of a longer sequence. So a null terminated UTF-8 string is simple - plugwash 2015-11-24 17:00
Well, good luck unpacking it if it has non-ascii. Just use Convert.ToBase64String - Erik Bergstedt 2015-12-12 10:30
Example demonstrating this does not terminate with null characters. Encoding.Ascii yields same result - Assimilater 2017-06-29 00:02
I receive this error when trying your solution: there is no argument given that corresponds to the required formal parameter index of Encoding.GetString(bytes[],int, int - user3841581 2017-11-04 15:45


292

There're at least four different ways doing this conversion.

  1. Encoding's GetString
    , but you won't be able to get the original bytes back if those bytes have non-ASCII characters.

  2. BitConverter.ToString
    The output is a "-" delimited string, but there's no .NET built-in method to convert the string back to byte array.

  3. Convert.ToBase64String
    You can easily convert the output string back to byte array by using Convert.FromBase64String.
    Note: The output string could contain '+', '/' and '='. If you want to use the string in a URL, you need to explicitly encode it.

  4. HttpServerUtility.UrlTokenEncode
    You can easily convert the output string back to byte array by using HttpServerUtility.UrlTokenDecode. The output string is already URL friendly! The downside is it needs System.Web assembly if your project is not a web project.

A full example:

byte[] bytes = { 130, 200, 234, 23 }; // A byte array contains non-ASCII (or non-readable) characters

string s1 = Encoding.UTF8.GetString(bytes); // ���
byte[] decBytes1 = Encoding.UTF8.GetBytes(s1);  // decBytes1.Length == 10 !!
// decBytes1 not same as bytes
// Using UTF-8 or other Encoding object will get similar results

string s2 = BitConverter.ToString(bytes);   // 82-C8-EA-17
String[] tempAry = s2.Split('-');
byte[] decBytes2 = new byte[tempAry.Length];
for (int i = 0; i < tempAry.Length; i++)
    decBytes2[i] = Convert.ToByte(tempAry[i], 16);
// decBytes2 same as bytes

string s3 = Convert.ToBase64String(bytes);  // gsjqFw==
byte[] decByte3 = Convert.FromBase64String(s3);
// decByte3 same as bytes

string s4 = HttpServerUtility.UrlTokenEncode(bytes);    // gsjqFw2
byte[] decBytes4 = HttpServerUtility.UrlTokenDecode(s4);
// decBytes4 same as bytes
2014-02-28 02:31
by detale
LINQ it: var decBytes2 = str.Split('-').Select(ch => Convert.ToByte(ch, 16)).ToArray();drtf 2014-07-13 14:43


21

A general solution to convert from byte array to string when you don't know the encoding:

static string BytesToStringConverted(byte[] bytes)
{
    using (var stream = new MemoryStream(bytes))
    {
        using (var streamReader = new StreamReader(stream))
        {
            return streamReader.ReadToEnd();
        }
    }
}
2015-09-20 08:24
by Nir
But this assumes that there is either an encoding BOM in the byte stream or that it is in UTF-8. But you can do the same with Encoding anyway. It doesn't magically solve the problem when you don't know the encoding - Sebastian Zander 2017-09-26 17:05


12

Definition:

public static string ConvertByteToString(this byte[] source)
{
    return source != null ? System.Text.Encoding.UTF8.GetString(source) : null;
}

Using:

string result = input.ConvertByteToString();
2014-10-16 01:04
by Erçin Dedeoğlu


9

Converting a byte[] to a string seems simple but any kind of encoding is likely to mess up the output string. This little function just works without any unexpected results:

private string ToString(byte[] bytes)
{
    string response = string.Empty;

    foreach (byte b in bytes)
        response += (Char)b;

    return response;
}
2015-04-22 11:48
by AndrewJE
I received System.FormatException using your method when I unpacked it with Convert.FromBase64String - Erik Bergstedt 2015-12-12 10:20
@ AndrewJE this will take for even to compute if you have a large byte array like the one used from the pictures - user3841581 2017-11-04 16:55


8

Using (byte)b.ToString("x2"), Outputs b4b5dfe475e58b67

public static class Ext {

    public static string ToHexString(this byte[] hex)
    {
        if (hex == null) return null;
        if (hex.Length == 0) return string.Empty;

        var s = new StringBuilder();
        foreach (byte b in hex) {
            s.Append(b.ToString("x2"));
        }
        return s.ToString();
    }

    public static byte[] ToHexBytes(this string hex)
    {
        if (hex == null) return null;
        if (hex.Length == 0) return new byte[0];

        int l = hex.Length / 2;
        var b = new byte[l];
        for (int i = 0; i < l; ++i) {
            b[i] = Convert.ToByte(hex.Substring(i * 2, 2), 16);
        }
        return b;
    }

    public static bool EqualsTo(this byte[] bytes, byte[] bytesToCompare)
    {
        if (bytes == null && bytesToCompare == null) return true; // ?
        if (bytes == null || bytesToCompare == null) return false;
        if (object.ReferenceEquals(bytes, bytesToCompare)) return true;

        if (bytes.Length != bytesToCompare.Length) return false;

        for (int i = 0; i < bytes.Length; ++i) {
            if (bytes[i] != bytesToCompare[i]) return false;
        }
        return true;
    }

}
2014-11-22 17:15
by metadings


5

There is also class UnicodeEncoding, quite simple in usage:

ByteConverter = new UnicodeEncoding();
string stringDataForEncoding = "My Secret Data!";
byte[] dataEncoded = ByteConverter.GetBytes(stringDataForEncoding);

Console.WriteLine("Data after decoding: {0}", ByteConverter.GetString(dataEncoded));
2015-05-18 13:38
by P.K.
But not UTF-8 methinks - david.pfx 2015-07-14 10:36
UnicodeEncoding is the worst class name ever; unicode isn't an encoding at all. That class is actually UTF-16. The little-endian version, I think - Nyerguds 2016-11-17 08:16


2

Alternatively:

 var byteStr = Convert.ToBase64String(bytes);
2016-09-15 05:55
by Fehr


2

A Linq one-liner for converting a byte array byteArrFilename read from a file to a pure ascii C-style zero-terminated string would be this: Handy for reading things like file index tables in old archive formats.

String filename = new String(byteArrFilename.TakeWhile(x => x != 0)
                              .Select(x => x < 128 ? (Char)x : '?').ToArray());

I use '?' as default char for anything not pure ascii here, but that can be changed, of course. If you want to be sure you can detect it, just use '\0' instead, since the TakeWhile at the start ensures that a string built this way cannot possibly contain '\0' values from the input source.

2016-11-17 08:12
by Nyerguds


2

BitConverter class can be used to convert a byte[] to string.

var convertedString = BitConverter.ToString(byteAttay);

Documentation of BitConverter class can be fount on MSDN

2017-01-05 10:53
by Sagar
This converts the byte array to a hexadecimal string representing each byte, which is generally not what you want when converting bytes to a string. If you do, then that's another question, see for example How do you convert Byte Array to Hexadecimal String, and vice versa? - CodeCaster 2017-01-05 10:59
Not what OP aske - Winter 2017-07-19 13:46


2

To my knowledge none of the given answers guarantee correct behavior with null termination. Until someone shows me differently I wrote my own static class for handling this with the following methods:

// Mimics the functionality of strlen() in c/c++
// Needed because niether StringBuilder or Encoding.*.GetString() handle \0 well
static int StringLength(byte[] buffer, int startIndex = 0)
{
    int strlen = 0;
    while
    (
        (startIndex + strlen + 1) < buffer.Length // Make sure incrementing won't break any bounds
        && buffer[startIndex + strlen] != 0       // The typical null terimation check
    )
    {
        ++strlen;
    }
    return strlen;
}

// This is messy, but I haven't found a built-in way in c# that guarentees null termination
public static string ParseBytes(byte[] buffer, out int strlen, int startIndex = 0)
{
    strlen = StringLength(buffer, startIndex);
    byte[] c_str = new byte[strlen];
    Array.Copy(buffer, startIndex, c_str, 0, strlen);
    return Encoding.UTF8.GetString(c_str);
}

The reason for the startIndex was in the example I was working on specifically I needed to parse a byte[] as an array of null terminated strings. It can be safely ignored in the simple case

2017-06-29 00:19
by Assimilater
Mine does, actually. byteArr.TakeWhile(x => x != 0) is a quick and easy way to solve the null termination problem - Nyerguds 2017-09-21 09:11


0

Try this:

string myresult = System.Text.Encoding.UTF8.GetString(byteArray);
2018-02-23 00:43
by Bill


0

hier is a result where you didnt have to bother with encoding. I used it in my network class and send binary objects as string with it.

        public static byte[] String2ByteArray(string str)
        {
            char[] chars = str.ToArray();
            byte[] bytes = new byte[chars.Length * 2];

            for (int i = 0; i < chars.Length; i++)
                Array.Copy(BitConverter.GetBytes(chars[i]), 0, bytes, i * 2, 2);

            return bytes;
        }

        public static string ByteArray2String(byte[] bytes)
        {
            char[] chars = new char[bytes.Length / 2];

            for (int i = 0; i < chars.Length; i++)
                chars[i] = BitConverter.ToChar(bytes, i * 2);

            return new string(chars);
        }
2018-07-06 13:27
by Marco Pardo
Where's your unit test ;) - Minh Tran 2018-09-16 18:10
didnt have one. But this function is in use for binary transmission in our company-network and so far 20TB were re- and encoded correctly. So for me this function works : - Marco Pardo 2018-09-17 19:10


0

In adition to the selected answer, if you're using .NET35 or .NET35 CE, you have to specify the index of the first byte to decode, and the number of bytes to decode:

string result = System.Text.Encoding.UTF8.GetString(byteArray,0,byteArray.Length);
2019-02-01 19:55
by The One
Ads