XPCOM字符串操作（二）

 

Unicode转化ns*CString vs. ns*String

字符串有两种基本的存储格式: 8-bit code unit (byte/char) 字符串, 或者 16-bit code unit (PRUnichar) 字符串. 所有带大写C开头的字符串类都是8-bit的. 包括include nsCString, nsDependentCString, 当然所有不含大写C开头的字符串类都是16字节的。

8-bit的字符串可以使用N多编码，但是16-bit的字符串永远都是使用UTF-16编码。最常见的编码格式如下：

• ASCII - 8-bit encoding for basic English-only strings. Each ASCII value is stored in exactly one byte in the array.
• UCS2 - 16-bit encoding for a subset of Unicode, BMP. The Unicode value of a character stored in UCS2 is stored in exactly one 16-bit PRUnichar in a string class.
• UTF-8 - 8-bit encoding for Unicode characters. Each Unicode characters is stored in up to 4 bytes in a string class. UTF-8 is capable of representing the entire Unicode character repertoire, and it efficiently maps to UTF-32.
• UTF-16 - 16-bit encoding for Unicode storage, backwards compatible with UCS2. The Unicode value of a character stored in UTF-16 may require one or two 16-bitPRUnichars in a string class. The contents of nsAString always has to be regarded as in this encoding instead of UCS2. UTF-16 is capable of representing the entire Unicode character repertoire, and it efficiently maps to UTF-32. (Win32 W APIs and Mac OS X natively use UTF-16.)

当需要进行转码的时候， 最重要的是先确定你当前的操作的字符串的宾吗，然后再决定使用正确的转码机制：

When writing new code, it can be confusing to know which storage class and encoding is the most appropriate. There is no single answer to this question, but there are a few important guidelines:

• Is the string always ASCII? First and foremost, you need to determine what kinds of values will be stored in the string. If the strings are always internal, ASCII strings such as "left", "true", "background" and so forth, then straight C-strings are probably the way to go.
• If the string is ASCII, will it be compared to, assigned to, or otherwise interact with non-ASCII strings? When assigning or comparing an 8-bit ASCII value (in)to a 16-bit UCS2 string, an "inflation" needs to happen at runtime. If your strings are small enough (say, less than 64 bytes) then it may make sense to store your string in a 16-bit unicode class as well, to avoid the extra conversion. The tradeoff is that your ASCII string takes up twice as much space as a 16-bit Unicode string than it would as an 8-bit string.
• Is the string usually ASCII, but needs to support unicode? If your string is most often ASCII but needs to be able to store Unicode characters, then UTF-8 may be the right encoding. ASCII characters will still be stored in 8-bit storage but other Unicode characters will take up 2 to 4 bytes. However if the string ever needs to be compared or assigned to a 16-bit string, a runtime conversion will be necessary.
• Are you storing large strings of non-ASCII data? Up until this point, UTF-8 might seem like the ideal encoding. The drawback is that for most non-European characters (such as Chinese, Indian and Japanese) in BMP, UTF-8 takes 50% more space than UTF-16. For characters in plane 1 and above, both UTF-8 and UTF-16 take 4 bytes.
• Do you need to manipulate the contents of a Unicode string? One problem with encoding Unicode characters in UTF-8 or other 8-bit storage formats is that the actual Unicode character can span multiple bytes in a string. In most encodings, the actual number of bytes varies from character to character. When you need to iterate over each character, you must take the encoding into account. This is vastly simplified when iterating 16-bit strings because each 16-bit code unit (PRUnichar) corresponds to a Unicode character as long as all characters are in BMP, which is often the case. However, you have to keep in mind that a single Unicode character in plane 1 and beyond is represented in two 16-bit code units in 16-bit strings so that the number of PRUnichar's is not always equal to the number of Unicode characters. For the same reason, the position and the index in terms of 16-bit code units are not always the same as the position and the index in terms of Unicode characters.

To assist with ASCII, UTF-8, and UTF-16 conversions, there are some helper methods and classes. Some of these classes look like functions, because they are most often used as temporary objects on the stack.

UTF-8 / UTF-16 conversion

NS_ConvertUTF8toUTF16(const nsACString&) - a nsAutoString subclass that converts a UTF-8 encoded nsACString or const char* to a 16-bit UTF-16 string. If you need a const PRUnichar* buffer, you can use the .get() method. For example:

/* signature: void HandleUnicodeString(const nsAString& str); */
object->HandleUnicodeString(NS_ConvertUTF8toUTF16(utf8String));
          
/* signature: void HandleUnicodeBuffer(const PRUnichar* str); */
object->HandleUnicodeBuffer(NS_ConvertUTF8toUTF16(utf8String).get());

NS_ConvertUTF16toUTF8(const nsAString&) - a nsCAutoString which converts a 16-bit UTF-16 string (nsAString) to a UTF-8 encoded string. As above, you can use.get() to access a const char* buffer.

/* signature: void HandleUTF8String(const nsACString& str); */
object->HandleUTF8String(NS_ConvertUTF16toUTF8(utf16String));

/* signature: void HandleUTF8Buffer(const char* str); */
object->HandleUTF8Buffer(NS_ConvertUTF16toUTF8(utf16String).get());

CopyUTF8toUTF16(const nsACString&, nsAString&) - converts and copies:

// return a UTF-16 value
void Foo::GetUnicodeValue(nsAString& result) {
    CopyUTF8toUTF16(mLocalUTF8Value, result);
 }

AppendUTF8toUTF16(const nsACString&, nsAString&) - converts and appends:

// return a UTF-16 value
void Foo::GetUnicodeValue(nsAString& result) {
    result.AssignLiteral("prefix:");
    AppendUTF8toUTF16(mLocalUTF8Value, result);
}

UTF8ToNewUnicode(const nsACString&, PRUint32* aUTF16Count = nsnull) - allocates and converts (the optional parameter will contain the number of 16-byte units upon return, if non-null):

void Foo::GetUTF16Value(PRUnichar** result) {
    *result = UTF8ToNewUnicode(mLocalUTF8Value);
}

CopyUTF16toUTF8(const nsAString&, nsACString&) - converts and copies:

// return a UTF-8 value
void Foo::GetUTF8Value(nsACString& result) {
    CopyUTF16toUTF8(mLocalUTF16Value, result);
}

AppendUTF16toUTF8(const nsAString&, nsACString&) - converts and appends:

// return a UTF-8 value
void Foo::GetUnicodeValue(nsACString& result) {
    result.AssignLiteral("prefix:");
    AppendUTF16toUTF8(mLocalUTF16Value, result);
}

ToNewUTF8String(const nsAString&) - allocates and converts:

void Foo::GetUTF8Value(char** result) {
    *result = ToNewUTF8String(mLocalUTF16Value);
}

Lossy Conversion

The following should only be used when you can guarantee that the original string is ASCII. These helpers are very similar to the UTF-8 / UTF-16 conversion helpers above.

UTF-16 to ASCII converters

These converters are very dangerous because they lose information during the conversion process. You should avoid UTF-16 to ASCII conversions unless your strings are guaranteed to be ASCII. Each 16-bit code unit in 16-bit string is simply cast to an 8-bit byte, which means all Unicode character values above 0xFF are converted to an arbitrary 8-bit byte.

• NS_LossyConvertUTF16toASCII(nsAString) - a nsCAutoString which holds a temporary buffer containing the deflated value of the string.
• LossyCopyUTF16toASCII(nsAString, nsACString) - does an in-place conversion from UTF-16 into an ASCII string object.
• LossyAppendUTF16toASCII(nsAString, nsACString) - appends an UTF-16 string to an ASCII string, losing non-ASCII values.
• ToNewCString(nsAString) - allocates a new char* string.

ASCII to UTF-16 converters

These converters are very dangerous because they will mangle any non-ASCII string into a meaningless UTF-16 string. You should avoid ASCII to UTF-16 conversions unless your strings are guaranteed to be ASCII. For instance, if you have an 8-bit string encoded in a multibyte character encoding, each byte of the string will be "inflated" to a 16-bit number by simple casting.

For example, imagine a UTF-8 string where the first Unicode character of the string is represented with a 3-byte UTF-8 sequence, the "inflated" UTF-16 string will contain the 3 PRUnichar's instead of the single PRUnichar that represents the first character. These PRUnichar's have nothing to do with the first Unicode character in the UTF-8 string.

• NS_ConvertASCIItoUTF16(nsACString) - a nsAutoString which holds a temporary buffer containing the inflated value of the string.
• CopyASCIItoUTF16(nsACString, nsAString) - does an in-place conversion from one string into a Unicode string object.
• AppendASCIItoUTF16(nsACString, nsAString) - appends an ASCII string to a Unicode string.
• ToNewUnicode(nsACString) - Creates a new PRUnichar* string which contains the inflated value.

Common Patterns

Callee-allocated Parameters

Many APIs result in a method allocating a buffer in order to return strings to its caller. This can be tricky because the caller has to remember to free the string when they have finished using it. Fortunately, the nsXPIDLString class makes this very easy.

A method may look like this:

void GetValue(PRUnichar** aValue)
{
    *aValue = ToNewUnicode(foo);
}

Without the string classes, the caller would need to free the string:

{
    PRUnichar* val;
    GetValue(&val);

    if (someCondition) {
        // don't forget to free the value!
        NS_Free(val);
        return NS_ERROR_FAILURE;
    }

    ...
    // and later, still don't forget to free!
    NS_Free(val);
}

With nsXPIDLString you never have to worry about this. You can just use getter_Copies() to wrap the string class, and the class will remember to free the buffer when it goes out of scope:

{
    nsXPIDLString val;
    GetValue(getter_Copies(val));

    // val will free itself here
    if (someCondition)
        return NS_ERROR_FAILURE;
    ...
    // and later, still nothing to free
}

The resulting code is much simpler, and easy to read.

Literal Strings

A literal string is a raw string value that is written in some C++ code. For example, in the statement printf("Hello World\n"); the value "Hello World\n" is a literal string. It is often necessary to insert literal string values when an nsAString or nsACString is required. These four macros will provide you with the necessary conversion:

• NS_LITERAL_CSTRING(literal string) - a temporary nsCString
• NS_NAMED_LITERAL_CSTRING(variable,literal string) - declares a nsCString variable named variable
• NS_LITERAL_STRING(literal string) - a temporary nsString with the unicode version of literal string
• NS_NAMED_LITERAL_STRING(variable,literal string) - declares a nsString variable named variable with the unicode version of literal string

The purpose of the CSTRING versions of these macros may seem unnecessary, given that nsDependentCString will also wrap a string value in an nsCString. The advantage to these macros is that the length of these strings is calculated at compile time, so the string does not need to be scanned at runtime to determine its length.

The STRING versions of these macros provide a portable way of declaring UTF-16 versions of the given literal string, avoiding runtime conversion on platforms which support literal UTF-16 strings (e.g., MSVC++ and GCC with the -fshort-wchar option).

// call Init(const PRUnichar*)
Init(L"start value"); // bad - L"..." is not portable!
Init(NS_ConvertASCIItoUTF16("start value").get()); // bad - runtime ASCII->UTF-16 conversion!
      
// call Init(const nsAString&)
Init(nsDependentString(L"start value")); // bad - not portable!
Init(NS_ConvertASCIItoUTF16("start value")); // bad - runtime ASCII->UTF-16 conversion!
      
// call Init(const nsACString&)
Init(nsDependentCString("start value")); // bad - length determined at runtime

Here are some examples of proper NS_LITERAL_[C]STRING usage.

// call Init(const PRUnichar*)
Init(NS_LITERAL_STRING("start value").get());

// call Init(const nsAString&)
Init(NS_LITERAL_STRING("start value"));

// call Init(const nsACString&)
Init(NS_LITERAL_CSTRING("start value"));

There are a few details which can be useful in tracking down issues with these macros:

NS_LITERAL_STRING does compile-time conversion to UTF-16 on some platforms (e.g. Windows, Linux, and Mac) but does runtime conversion on other platforms. By usingNS_LITERAL_STRING your code is guaranteed to use the best possible conversion for the platform in question.

Because some platforms do runtime conversion, the use of literal string concatenation inside a NS_LITERAL_STRING/NS_NAMED_LITERAL_STRING macro will compile on these platforms, but not on platforms which support compile-time conversion.

For example:

// call Init(nsAString&)
Init(NS_LITERAL_STRING("start "
     "value")); // only compiles on some platforms

The reason for this is that on some platforms, the L"..." syntax is used, but it is only applied to the first string in the concatenation ("start "). When the compiler attempts to concatenate this with the non-Unicode string "value" it gets confused.

Also, using preprocessor macros as the string literal is unsupported:

#define some_string "See Mozilla Run"
...
Init(NS_LITERAL_STRING( some_string )); // only compiles on some platforms/with some compilers.

String Concatenation

Strings can be concatenated together using the + operator. The resulting string is a const nsSubstringTuple object. The resulting object can be treated and referenced similarly to a nsAString object. Concatenation does not copy the substrings. The strings are only copied when the concatenation is assigned into another string object. ThensSubstringTuple object holds pointers to the original strings. Therefore, the nsSubstringTuple object is dependent on all of its substrings, meaning that their lifetime must be at least as long as the nsSubstringTuple object.

For example, you can use the value of two strings and pass their concatenation on to another function which takes an const nsAString&:

void HandleTwoStrings(const nsAString& one, const nsAString& two) {
    // call HandleString(const nsAString&)
    HandleString(one + two);
}

NOTE: The two strings are implicitly combined into a temporary nsString in this case, and the temporary string is passed into HandleString. If HandleString assigns its input into another nsString, then the string buffer will be shared in this case negating the cost of the intermediate temporary. You can concatenate N strings and store the result in a temporary variable:

NS_NAMED_LITERAL_STRING(start, "start ");
NS_NAMED_LITERAL_STRING(middle, "middle ");
NS_NAMED_LITERAL_STRING(end, "end");
// create a string with 3 dependent fragments - no copying involved!
nsString combinedString = start + middle + end;

// call void HandleString(const nsAString&);
HandleString(combinedString);

If you are using NS_LITERAL_STRING to create a temporary that is only used once, then it is safe to define it inside a concatenation because the string buffer will live as long as the temporary concatenation object (of type nsSubstringTuple).

// call HandlePage(const nsAString&);
// safe because the concatenated-string will live as long as its substrings
HandlePage(NS_LITERAL_STRING("start ") + NS_LITERAL_STRING("end"));

Local variables

Local variables within a function are usually stored on the stack. The nsAutoString/nsCAutoString classes are derivatives of the nsString/nsCString classes. They own a 64-character buffer allocated in the same storage space as the string itself. If the nsAutoString is allocated on the stack, then it has at its disposal a 64-character stack buffer. This allows the implementation to avoid allocating extra memory when dealing with small strings.

...
nsAutoString value;
GetValue(value); // if the result is less than 64 code units,
                 // then this just saved us an allocation
...

Member variables

In general, you should use the concrete classes nsString and nsCString for member variables.

class Foo {
    ...
    // these store UTF-8 and UTF-16 values respectively
    nsCString mLocalName;
    nsString mTitle;
};

Note that the strings are declared directly in the class, not as pointers to strings. Don't do this:

class Foo {
public:
    Foo() { 
        mLocalName = new nsCString(); 
        mTitle = new nsString(); 
    }
    ~Foo() { delete mLocalName; delete mTitle; }

private:
    // these store UTF-8 and UTF-16 values respectively
    nsCString* mLocalName;
    nsString*  mTitle;
};

The above code may appear to save the cost of the string objects, but nsString/nsCString are small objects - the overhead of the allocation outweighs the few bytes you'd save by keeping a pointer.

Another common incorrect pattern is to use nsAutoString/nsCAutoString for member variables. As described in Local Variables, these classes have a built in buffer that make them very large. This means that if you include them in a class, they bloat the class by 64 bytes (nsCAutoString) or 128 bytes (nsAutoString).

An example:

class Foo {
    ...
      
    // bloats 'Foo' by 128 bytes!
    nsAutoString mLocalName;
};

Raw Character Pointers

PromiseFlatString() can be used to create a temporary buffer which holds a null-terminated buffer containing the same value as the source string.PromiseFlatString() will create a temporary buffer if necessary. This is most often used in order to pass an nsAString to an API which requires a null-terminated string.

In the following example, an nsAString is combined with a literal string, and the result is passed to an API which requires a simple character buffer.

// Modify the URL and pass to AddPage(const PRUnichar* url)
void AddModifiedPage(const nsAString& url) {
    NS_NAMED_LITERAL_STRING(httpPrefix, "http://");
    const nsAString& modifiedURL = httpPrefix + url;

    // creates a temporary buffer
    AddPage(PromiseFlatString(modifiedURL).get());
}

PromiseFlatString() is smart when handed a string that is already null-terminated. It avoids creating the temporary buffer in such cases.

// Modify the URL and pass to AddPage(const PRUnichar* url)
void AddModifiedPage(const nsAString& url, PRBool addPrefix) {
    if (addPrefix) {
        // MUST create a temporary buffer - string is multi-fragmented
        NS_NAMED_LITERAL_STRING(httpPrefix, "http://");
        AddPage(PromiseFlatString(httpPrefix + modifiedURL));
    } else {
        // MIGHT create a temporary buffer, does a runtime check
        AddPage(PromiseFlatString(url).get());
    }
}

printf and a UTF-16 string

For debugging, it's useful to printf a UTF-16 string (nsString, nsAutoString, nsXPIDLString, etc). To do this usually requires converting it to an 8-bit string, because that's what printf expects. However, on Windows, the following should work:

printf("%S\n", yourString.get());

(Note: I didn't test this. Also, I'm not sure what exactly this does to non-ASCII characters, especially when they are outside the system codepage). The reason that this doesn't work on Unix is because a wchar_t, which is what %S expects, is usually 4 bytes there (even when Mozilla is compiled with -fshort-wchar, because this would require libc to be compiled with -fshort-wchar).

If non-ASCII characters aren't important, use:

printf("%s\n", NS_LossyConvertUTF16toASCII(yourString).get());

On platforms that use UTF-8 for console output (most Linux distributions), this works:

printf("%s\n", NS_ConvertUTF16toUTF8(yourString).get());

IDL

The string library is also available through IDL. By declaring attributes and methods using the specially defined IDL types, string classes are used as parameters to the corresponding methods.

IDL String types

The C++ signatures follow the abstract-type convention described above, such that all method parameters are based on the abstract classes. The following table describes the purpose of each string type in IDL.

IDL type C++ Type Purpose

string	char*	Raw character pointer to ASCII (7-bit) string, no string classes used. High bit is not guaranteed across XPConnect boundaries.
wstring	PRUnichar*	Raw character pointer to UTF-16 string, no string classes used.
AString	nsAString	UTF-16 string.
ACString	nsACString	8-bit string. All bits are preserved across XPConnect boundaries.
AUTF8String	nsACString	UTF-8 string. Converted to UTF-16 as necessary when value is used across XPConnect boundaries.
DOMString	nsAString	UTF-16 string type used in the DOM. The same as AString with a few odd XPConnect exceptions: When the special JavaScript value null is passed to a DOMString parameter of an XPCOM method, it becomes a void DOMString. The special JavaScript value undefined becomes the string "undefined".

C++ Signatures

In IDL, in parameters are read-only, and the C++ signatures for *String parameters follows the above guidelines by using const nsAString& for these parameters. outand inout parameters are defined simply as nsAString so that the callee can write to them.

IDL C++

interface nsIFoo : nsISupports { attribute AString utf16String; AUTF8String getValue(in ACString key); };	class nsIFoo : public nsISupports { NS_IMETHOD GetUtf16String(nsAString& aResult) = 0; NS_IMETHOD SetUtf16String(const nsAString& aValue) = 0; NS_IMETHOD GetValue(const nsACString& aKey, nsACString& aResult) = 0; };

In the above example, utf16String is treated as a UTF-16 string. The implementation of GetUtf16String() will use aResult.Assign to "return" the value. InSetUtf16String() the value of the string can be used through a variety of methods including Iterators, PromiseFlatString, and assignment to other strings.

In GetValue(), the first parameter, aKey, is treated as a raw sequence of 8-bit values. Any non-ASCII characters in aKey will be preserved when crossing XPConnect boundaries. The implementation of GetValue() will assign a UTF-8 encoded 8-bit string into aResult. If the this method is called across XPConnect boundaries, such as from a script, then the result will be decoded from UTF-8 into UTF-16 and used as a Unicode value.

Choosing a string type

It can be difficult to determine the correct string type to use for IDL. The following points should help determine the appropriate string type.

• Using string classes may avoid new memory allocation for out parameters. For example, if the caller is using an nsAutoString to receive the value for an outparameter, (defined in C++ as simply nsAString& then assignment of short (less than 64-characters) values to an out parameter will only copy the value into thensAutoString's buffer. Moreover, using the string classes allows for sharing of string buffers. In many cases, assigning from one string object to another avoids copying in favor of simply incrementing a reference count.
• in strings using string classes often have their length pre-calculated. This can be a performance win.
• In cases where a raw-character buffer is required, string and wstring provide faster access than PromiseFlatString.
• UTF-8 strings defined with AUTF8String may need to be decoded when crossing XPConnect boundaries. This can be a performance hit. On the other hand, UTF-8 strings take up less space for strings that are commonly ASCII.
• UTF-16 strings defined with wstring or AString are fast when the unicode value is required. However, if the value is more often ASCII, then half of the storage space of the underlying string may be wasted.

String Guidelines

Follow these simple rules in your code to keep your fellow developers, reviewers, and users happy.

• Avoid *WithConversion functions at all costs: AssignWithConversion, AppendWithConversion, EqualsWithConversion, etc
• Use the most abstract string class that you can. Usually this is:
  • nsAString for function parameters
  • nsString for member variables
  • nsAutoString or nsXPIDLString for local (stack-based) variables
• Use NS_LITERAL_[C]STRING / NS_NAMED_LITERAL_[C]STRING to represent literal strings (i.e. "foo") as nsAString-compatible objects.
• Use string concatenation (i.e. the "+" operator) when combining strings.
• Use nsDependentString when you have a raw character pointer that you need to convert to an nsAString-compatible string.
• Use Substring() to extract fragments of existing strings.
• Use iterators to parse and extract string fragments.

Appendix A - What class to use when

This table provides a quick reference for what classes you should be using.

Context class Notes

Local Variables	nsAutoString nsCAutoString
Class Member Variables	nsString nsCString
Method Parameter types	nsAString nsACString	Use abstract classes for parameters. Use const nsAString& for "in" parameters and nsAString& for "out" parameters.
Retrieving "out" string/wstrings	nsXPIDLString nsXPIDLCString	Use getter_Copies(). Similar to nsString / nsCString.
Wrapping character buffers	nsDependentString nsDependentCString	Wrap const char* / const PRUnichar* buffers.
Literal strings	NS_LITERAL_STRING NS_LITERAL_CSTRING	Similar to nsDependent[C]String, but pre-calculates length at build time.

Appendix B - nsAString Reference

Read-only methods.

• Length()
• IsEmpty()
• IsVoid() - XPConnect will convert void nsAStrings to JavaScript null.
• BeginReading(iterator)
• EndReading(iterator)
• Equals(string[, comparator])
• First()
• Last()
• CountChar()
• Left(outstring, length)
• Mid(outstring, position, length)
• Right(outstring, length)
• FindChar(character)

Methods that modify the string.

• Assign(string)
• Append(string)
• Insert(string)
• Cut(start, length)
• Replace(start, length, string)
• Truncate(length)
• SetIsVoid(true) - Make it null. XPConnect will convert void nsAStrings to JavaScript null.
• BeginWriting(iterator)
• EndWriting(iterator)
• SetCapacity()