Sunday, February 19, 2017

Safer S...

I want, of course, to talk about "Safer Strings" today.

TL;DR: Add /DwxNO_UNSAFE_WXSTRING_CONV=1 to your compiler options today.

wxWidgets has had implicit conversion of wxString to const char* since the dawn of time (or about 1992, at any rate). This was always dangerous, as it allowed someone to accidentally write:

void show_and_free(const char* p) { ...; free(p); }
wxString s("...");
with catastrophic consequences, but such situations were relatively rare and it was thought that the convenience of having this implicit conversion overweighted the dangers. This is also why when we added Unicode support later, we also added implicit conversion to const wchar_t* and, when we added "STL" build mode, in which interoperability with the standard library is increased further even at the price of backwards compatibility, we added implicit conversions to std::string and std::wstring as well.

Unfortunately, with the merge of ANSI and Unicode build modes in wxWidgets 3, another, much more dangerous, problem has appeared because in the new combined mode we can now have a string containing Unicode characters not representable in the current locale encoding. And converting such strings to either char* or std::string inevitably results in a loss of data in this case, e.g.

double convert_temperature_to_celsius(const char* p) {
    const char* end;
    double t = strtod(p, &end);
    return 5.*(t - 32)/9.;

wxString s = wxGetTextFromUser("Enter temperature");
could, confusingly, result in always returning -17.77777, corresponding to 0°F, if the user decided to terminate the temperature entry with "°F" to explicitly indicate the scale used and the current encoding couldn't represent the degree symbol (which is the case of e.g. CP1250 under Microsoft Windows). In this case, conversion of wxString to char* would fail and p would be just empty.

Of course, this wouldn't happen if the code just used wxString::ToDouble() directly, or used wxChar and wxStrtod(), or used UTF-8, capable of representing any Unicode character, as encoding (which is practically always the case under Unix systems nowadays). So there are a lot of ways to write this code correctly, but, unfortunately, it was still too simple to write it wrongly accidentally lose the data entered by the user in this case. Clearly, implicit conversions potentially losing data are a bad idea, but we couldn't just turn them off in wxWidgets 3, as it would have broken almost all the existing programs which, empirically, all used these conversions in many places.

For the same reason, we still won't be able to turn this conversion off by default, even in wxWidgets 3.2. However now we at least provide a way to opt-in into safer behaviour. The arguably less interesting part of the changes is that you can now change the value of the compile-time wxUSE_UNSAFE_WXSTRING_CONV option when building the library. It is set to 1 by default, for compatibility, but if you build wxWidgets for the use in your own project, you are strongly advised to set it to 0 to permanently disable the unsafe, in the sense described above, implicit conversions.

Many people, however, don't build their own library, but use the one provided by their package manager under Unix/macOS or download our MSW binaries. These official binaries will continue to provide the unsafe conversions for compatibility, but you can define wxNO_UNSAFE_WXSTRING_CONV when building your own project to disable their use in your code without rebuilding the library. This symbol can be just #define'd before including any wxWidgets headers, but it is better to define it globally, in the compiler options in your make- or project file: just add /DwxNO_UNSAFE_WXSTRING_CONV=1 to it. And the main point of this long post is to convince you that you NEED TO DO just that: please define wxNO_UNSAFE_WXSTRING_CONV for your code and fix the resulting compilation errors to ensure that you don't lose any data entered by the user.

Fixing the compilation errors will, generally speaking, involve doing one of two things:

  • Either stop using char* (or std::string in the STL build) entirely and use wxString directly.
  • Or convert it to wchar_t* (or std::wstring) or convert wxString to UTF-8 encoding which will never lose data, using methods such as utf8_str(), which is a convenient synonym for mb_str(wxConvUTF8), or ToStdString(wxConvUTF8).
Of course, if you really need to use the current locale encoding, e.g. because you call a C standard library function using it, you will still need to perform the conversion to it, using just plain mb_str() and there will still be a possibility of the conversion to it failing, but at least now it won't happen implicitly.

Thanks for reading all this and, if you jumped to the end, hoping to quickly find the conclusion instead of reading this wall of text, please see the conclusion in the very beginning!

Monday, June 13, 2016

How to Keep a Secret

If your program needs to ask the user for a password, e.g. to connect to a web site or a database, chances are that it proposes a way to remember this password and not have to enter it the next time. This is convenient for the users, but is quite difficult to implement in any more or less secure way and a lot of programs end up storing the passwords in plain text, or something almost indistinguishable from it, e.g. base64-encoded string, in wxConfig.

But now wxWidgets provides a better way to do it with the new wxSecretStore class. It is still as simple to use as wxConfig but uses the OS-provided password storage facility such as Microsoft Windows credentials vault or OS X keychain, for storing the secrets you confide to it. Here is how you would normally use it:

Currently there is not much more that can be done with this class, the only functionality not illustrated by this example is deleting a previously stored secret, but in the future we could extend it, notably to provide a way to also ask the user to enter the password using the OS-provided dialog. Let us know if you use wxSecretStore and if you see possible improvements, please don't keep them secret!

Friday, April 01, 2016

Improving Inherently Important Internationalisation Issues

One of the lesser-known advantages of wxWidgets compared to many other libraries is that, using its API, you can centre a window on the screen and set its colour to grey without having to Americanise your programme. But, as natural as this behaviour is, and in spite of agreeing to wxWidgets licence (the very spelling of which should have been a sufficient hint), some users still expressed their surprise in their dialogue with us and refused our advice to accept using this flavour of the API. And so, to accommodate them, wxWidgets has traditionally provided alternatives such as Center() method, wxColor class and wxGRAY constant.

Admittedly, in practice this has been sufficient to avoid any problems for a long time while still remaining true to wxWidgets historically British roots. However, in addition to being British, wxWidgets was always mostly a European effort and, besides, knowing that it was actually born in Edinburgh, it might well end being exclusively European and not British at all in the near future (but don't worry, even if the worst happens, we plan to continue supporting British spelling for at least two more release cycles, in accordance with the general backwards compatibility policy). Considering this, it seems especially strange to have alternative American versions of the spelling, but no attention being paid to the other European languages.

And now, after releasing 3.1.0, it's finally time to do something about it as, clearly, there are few unresolved issues of comparable importance left. To begin with, I obviously had to add wxCouleur class which should undoubtedly help with wxWidgets adoption in francophone countries. But adding just the French variant would certainly be undiplomatic, so another pull request adds, in the grand European tradition of Franco-German alliance, wxFarbe, and both of them will be merged together to avoid any questions of precedence. Further, France and Germany notwithstanding, I certainly don't plan to discriminate against other countries but, in the usual spirit of Open Source, additions of wxColore, wxFarve, wxKleur etc will have to wait until the appropriate patches are submitted (luckily for them, Spanish users can, in the meanwhile, already reuse the American variant, just as they already do with many other words, e.g. burrito, for the classic American food).

Of course, nothing is ever as simple as planned and, while working on this, I quickly realized that Greek might present some practical difficulties as UTF-8 support is still, even in 2016, not perfect on some of the legacy platforms supported by wxWidgets, but it would be difficult to use wxΧρώμα without it. A compromise solution that we're currently discussing would be to introduce a wxChroma class right now and deprecate it by the time 3.2 is released as Unicode support should really be widely available everywhere by then (3.2 release is currently planned for the early 2030-ies).

I do hope you will enjoy these changes, but, let's be honest, this is just the beginning and we clearly still have a lot of work to do in this area. Fully supporting colour internationalization is important, but it's just the first step and we plan to go much further. Just imagine to be able to open a wxFenêtre and choose a wxStylo for drawing on it and then, perhaps if you live in Switzerland, create a wxKnopf as its child. The possibilities are endless! Unfortunately, the same can't be said about our resources, so we count on your help to make wxWidgets the best internationalized library of all time -- and we're looking forward to your contributions!

Monday, February 29, 2016

An unexpectedly expected release: 3.1.0 is available

Surprisingly, at least to me, we managed to make 3.1.0 release exactly as planned, i.e. today, without any delays. The price for this is that there are a couple of known problems in this release (see the errata on the release page), but we decided that it was not worth delaying the release for them.

This is not the most exciting release in wxWidgets project history, but it's still very much worth upgrading to as there has been a huge number of bug fixes in it, especially in wxGTK3 and wxOSX ports. Of course, as usual, there are a few new features as well, see the web site for a brief list and the change log for a fuller one and I'll try to find some time to write in more details about one of them, wxNativeWindow here in the near future.

The next goal is to release 3.0.3 to make at least some of the bug fixes, if not the new features, from 3.1.0 available to people using the system packages under Linux or just too cautious to start using the "development" releases (even although in my opinion this is not really justified as, on average, 3.1 branch has fewer, not more, bugs than 3.0.x one). As for 3.1.1, this will depend on how quickly the changes accumulate in the master, we'll see how it goes. In the meanwhile, we hope that you'll enjoy the new release!

Thursday, February 04, 2016

What g++ binaries for 3.1.0 release?

We plan to provide g++ binaries for the upcoming release, but we're not sure what are the versions and the build options that people would be interested in, so here is a quick poll to gather some information about this:


Please select 1-3 options corresponding to the binaries that would be useful to you to allow us to make the best choice. Thanks in advance!

Monday, February 01, 2016

3.1.0 Is Coming

Although we've been talking about it since quite some time, this time we're really going to make a new release soon. To be even more precise, I firmly expect to make it this month. Of course, I did wait until January was over to announce it. On the other hand, I did not wait for the February to be over neither even though it's the shortest month of the year. On still another hand, we did wait an extra year for this release, presumably just to have one extra day to make it in February of this year, as opposed to the last one... In any case, barring something really catastrophic happening, we should have a pre-release on February 15 and the release some time after that.

Of course, deciding to make the release is not quite enough, there are other things to do too, such as applying at least the most urgent and long standing patches and fixing selected bugs. So this week-end I spent some time doing this and, as the result, instead of 49 3.1.0-critical bugs we now have only 20 of them which is not as good as I hoped, but better than I expected. Of course, some of these bugs were just postponed, while others turned out to be not bugs at all after all or to have been already fixed. Still, quite a few real bugs and, notably, a few regressions which really couldn't remain before the release, were fixed as well, so there is that. I hope to deal with the remaining bugs during the next week and maybe even apply some non 3.1.0-targetted patches too, but in the worst case we could just postpone all the remaining 3.1.0 tickets as there is nothing absolutely critical there (and only a couple I'd characterise as being moderately critical).

So things look cautiously good for 3.1.0 and if we make it according to the plan, thus proving that our new release scripts, updated after the switch to git, work, hopefully 3.0.3 will follow soon afterwards.

Thursday, January 21, 2016

Dropping Carbon support under OS X

We would like to remove the old Mac OS X port using Carbon in the upcoming 3.1.0 release because we think nobody should be using it any longer (and we definitely know that nobody building 64 bit applications does because Carbon just doesn't exist in 64 bits) and it would make things simpler for us. But "should" and "is" are different things and if there are more than a handful of people who are still using Carbon and can't migrate to wxOSX/Cocoa port (I'd like to know why!), but, at the same time, still plan to update to 3.2 in the observable future, we could reconsider this. So if you'd like Carbon to continue to be supported, please let us know, in the comments here or on the mailing list. Thanks!