Few questions regarding setting up locales

[PatRoy]

Hello,

I'm writing a script that will handle localisation of my site for multi-languages. It will utilize either gettext() or, if not available, JSON files for translations ...

I have a few questions which I'm not sure about:

1. For now, I'm only calling the putenv() and setlocale() functions inside my _initLocaleGettext(). Should I also call these even if I'm using JSON files (if gettext not installed/configured in php.ini), say in my _initLocaleJSON()?

 

2. When calling the lather functions:

// Bare in mind that self::$_lang holds the desired language, like: 'en_US', 'fr_FR' or 'nl_NL'.
putenv('LANG=' . self::$_lang);
setlocale(LC_ALL, self::$_lang);

Should I specify the codeset as well in the putenv() AND/OR setlocale() functions (i.e. en_US.utf8 or en_US.iso88591) ? I've noticed that some servers don't necessarily have just 'en_US'.  My box for instance had just 'en_US.utf8' in the output of: locale -a

 

3. Gettext will only work if I'm using a locale that's installed on the server (locale -a). Would this be a proper way of checking my desired locale is installed?

if ( setlocale(LC_ALL, self::$_lang) === false ) throw new Exception("Locale '" . self::$_lang . "' is not installed on the server!");

 

The following is what I had before, but REALLY don't think it's the appropriate way, because a) it's an OS dependent program, and b) calls for an execution of a program on the system every time a page loads..

$available_system_locales = explode(PHP_EOL, shell_exec('locale -a'));
$working_locale = self::$_lang . '.' . strtolower(str_replace('-', '', self::$_codeset));
if ( ! in_array($working_locale, $available_system_locales) ) throw new Exception("Locale '$working_locale' is not installed on the server!");

 

4.  This question could be another topic, but while I'm at it:

My JSON files (one file per 'domain') will be like the following 2 examples:

Example JSON French file: 
{
  "": {
    "domain": "prestadesk",
    "language": "fr_FR",
    "plural-forms": "nplurals=2; plural=(n > 1);"
  },

  "Welcome, %s!": "Bienvenu, %s!",
  "This page will show the dashboard": "Cette page affichera le tableau de bord",

  "Only one unread message": "Vous avez qu'un seul message",
  "%d unread messages": [
    "Vous avez %d messages"
  ]
}

Example JSON Serbe file:
{
  "": {
    "domain": "prestadesk",
    "language": "sr_CS",
    "Plural-Forms": "nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2);"
  },

  "Welcome, %s!": "Dobrodosli, %s!",
  "This page will show the dashboard": "Ova stranica ce prikazati kontrolnu tablu",

  "I wrote a line of code": "Napisao sam liniju koda",
  "I wrote %d lines of code": [
    "Napisao sam %d liniju koda",
    "Napisao sam %d linije koda",
    "Napisao sam %d linija koda"
  ]
}

 

My 'gettext* like' functions have the same arguments has the original gettext(). For now, all I'm doing in order to find out if I use the singular or plural version of translation is checking if ($n > 1). You see I'm keeping gettext's standard plural-forms which are widely available. I would like to eventually parse these "plural-forms" / "plural" values to run each of the test cases, and match the succeeded one with the right plural array element. 

Is this crazy thinking? Can this be done fairly easily? I've yet to study the plural forms possibilities from http://docs.translatehouse.org/projects/localization-guide/en/latest/l10n/pluralforms.html. Not sure yet why has some () in them, some don't...  when there's multiple, then I think each test case ends with a '? X' indicating (I think?) the array element id to use for that test case... and (I think?) the last test case when multiple has 'no test case', just tells to use the last array ID for any other plural message...

 

I'll stop writting now. loll Thanks a million for your help and input!

Pat

 

 

[requinix]

First, as someone who hasn't seen your other threads:

How good does this need to be? Do you just want messages changed? Do you care about numbers? Currencies? Dates?

[PatRoy]

1 hour ago, requinix said:

First, as someone who hasn't seen your other threads:

How good does this need to be? Do you just want messages changed? Do you care about numbers? Currencies? Dates?

For now, I just need the messages changed. However, I never know down the road. PLUS I'm trying to create a versatile package which I can re-use in many projects.. Thus also one of the reasons my class can use either one, Gettext or JSON files... 

And on top, I want to do '"things right" the first time

 

Edited April 1, 2020 by PatRoy

[requinix]

Planning for the future is definitely good, but the thing is that localization can be a real pain to set up. How much time will you be spending working on something that you don't need right now? It's okay to do part of something now and the rest of it later. Delaying parts you don't need now also gives you time to learn about what your requirements will really be, instead of assuming now what you'll need in the future.

So I suggest you focus on just the message part for now.

String messages are pretty easy to solve in a rather generic way by using formatting and placeholders. You're gearing up for this really powerful but complicated solution when there are easier means available. Not everything has to be handled by this system.

Consider "you have X unread messages":
- Localized it will always look about the same - a simple sentence
- The "messages" word could be singular or plural, but there are languages where singular vs. plural is not as simple as adding a letter, or even as simple as designating a "singular" and "plural" form for a word
- Using %d as a placeholder works - unless you decide that you want to do something like write "you have no unread messages"

There are varying degrees of complexity to a possible implementation but you don't have to cater to the least common denominator for every single thing. You can do an 80/20 solution: make it simple for 80% of the use cases, more complicated for the other 20%.

<?php // messages/en.php

return [
	// a simple string template that will support a lot of different messages
	"You have %d foo thing(s)" => "You have %d foo thing(s)",

	// not good enough? use a function
	"You have %d bar thing%s" => function($n) {
		if ($n == 0) {
			return "You have no bar things";
		} else if ($n == 1) {
			return "You have 1 bar thing";
		} else {
			return sprintf("You have %d bar things");
		}
	}
];

function translate($template, ...$args) {
	$messages = get_translation_table();
	$template = $messages[$template] ?? $template;

	if ($template instanceof \Closure) {
		return $template(...$args);
	} else {
		return vsprintf($template, $args);
	}
}

And done.

[PatRoy]

On 4/2/2020 at 1:48 AM, requinix said:

Planning for the future is definitely good, but the thing is that localization can be a real pain to set up. How much time will you be spending working on something that you don't need right now? It's okay to do part of something now and the rest of it later. Delaying parts you don't need now also gives you time to learn about what your requirements will really be, instead of assuming now what you'll need in the future....

Hello, Requinix!

First thanks for your reply and piece of code example! Things have been a little chaotic around here (where is it not? ;)) and I've been a little offline.

I hear what you are saying about not having to 'code it all' right away, stick to whats really needed for needs. Totally ! If this project was a paid one, for a company, then yeah I'd do it like you say, cause you do have deadlines to respect, etc.

However, this is just a personal project of mine, and since I've got no work for now (and no life  ), it becomes a personal challenge, and a great learning tool! I've never programmed in PHP before. Used to code a lot in Java, VB (yeah' I'm that old   ), Perl, etc.. and worked quite a bit as a *Nix administrator, but I've quit all of that with a career change. I haven't really coded in nearly 12 years or so...  Thus, need to re-learn a lot

That being said, this hole issue initially started because I couldn't get gettext() to work on my server (because it's old NAS), and I really want to use this solution if I ever push my site to 'production' on a hosting provider. So, I decided to create a 'wrapper' class around gettext that would have a boolean set in its constructor to $useGettext true, or false. If true, well all of its functions uses gettext to get the messages. If false, then it uses JSON files...

 

I'm pretty happy now about it. It is super flexible thus will allow using translations on all kinds of hosts! If using JSON files, if also deals with plurals, just as Gettext does with a 'nplurals' and 'plural' value set, same syntax as gettext. and It's even able to handle loading additional domains (a.k.a translation files). Finally, I've made it so that all of its functions to translate messages are the same names as the gettext's functions, like so:

 

 *   Locale::gettext()   or Locale::_()    // Lookup a message in the current domain, singular form
 *   Locale::fgettext()  or Locale::_f()   // Lookup a message in the current domain, singular form, and sprintf's the message with $v value(s)
 *   Locale::ngettext()  or Locale::_n()   // Lookup a message in the current domain, plurial form
 *   Locale::fngettext   or Locale::_fn()  // Lookup a message in the current domain, plurial form, and sprintf's the message with $v value(s)
 *   Locale::dgettext()  or Locale::_d()   // Lookup a message in a given domain, singular form
 *   Locale::fdgettext   or Locale::_fd()  // Lookup a message in a given domain, singular form, and sprintf's the message with $v value(s)
 *   Locale::dngettext() or Locale::_dn()  // Lookup a message in a given domain, plurial form
 *   Locale::fdngettext  or Locale::_fdn() // Lookup a message in a given domain, plurial form, and sprintf's the message with $v value(s)

 

Do note that I know 'Locale' is already a used class name within PHP, but mine is in its own namespace: CorbeauPerdu\i18n\Locale

I'd love to know if anyone thinks this could be a real problem, and if so, I can just rename it

 

Anyhow, I'm writing all of this because I'm a strong believer in sharing, and wanted to share this class along.. (perhaps even create a project in Github or something). I'll repost link here if I do...

I'd love to share you the LocaleUsageExamples.php and the class itself, but can't attach any PHP in this forum. If you or anyone else is interested or curious, here's a temporary wetransfer link: https://wetransfer.com/downloads/eb670a961970210705a2824df897e43b20200416125315/ebbcf9d142a3eaa576eca92dc1638c7220200416125329/885d76

 

Thanks again for the time you've taken and your inputs!

Pat

[PatRoy]

5 hours ago, PatRoy said:

I'd love to share you the LocaleUsageExamples.php and the class itself, but can't attach any PHP in this forum. If you or anyone else is interested or curious, here's a temporary wetransfer link: https://wetransfer.com/downloads/eb670a961970210705a2824df897e43b20200416125315/ebbcf9d142a3eaa576eca92dc1638c7220200416125329/885d76

 

Alright, never mind the wetransfer BS. I've setup a Github repository:

https://github.com/ravenlost/CorbeauPerdu/tree/master/i18n/

 

While I was at it, I've also uploaded and shared my pretty sweet (I think!) DBWrapper as well: https://github.com/ravenlost/CorbeauPerdu/tree/master/Database/

I worked a long time on this DBWrapper as well, just because it annoyed me to re-write PDO code everytime...

Check out both of these classe's UsageExamples.php.  It shows it all

 

Edited April 16, 2020 by PatRoy
Bad URL links

[requinix]

Well, if you're just doing this for yourself then by all means, go ahead and experiment around with it.

10 minutes ago, PatRoy said:

I worked a long time on this DBWrapper as well, just because it annoyed me to re-write PDO code everytime...

Rewrite what? PDO is one of the best database APIs available...

[gizmola]

1 hour ago, requinix said:

Well, if you're just doing this for yourself then by all means, go ahead and experiment around with it.

Rewrite what? PDO is one of the best database APIs available...

Also there's DBAL which is part of Doctrine2.  Well to each their own. 

[PatRoy]

14 hours ago, requinix said:

Well, if you're just doing this for yourself then by all means, go ahead and experiment around with it.

Rewrite what? PDO is one of the best database APIs available...

By "rewrite", I don't mean rewriting PDO lolll !

I mean I don't like having to always do my new PDO(...) And then do my prepared statements etc...   I wanted to shorten my code !

Really, just have a look at the UsageExamples.php. it says it all way better than what I could explain

It's... As I said: a wrapper around PDO.

 

[PatRoy]

21 minutes ago, PatRoy said:

By "rewrite", I don't mean rewriting PDO lolll !

I mean I don't like having to always do my new PDO(...) And then do my prepared statements etc...   I wanted to shorten my code !

Really, just have a look at the UsageExamples.php. it says it all way better than what I could explain

It's... As I said: a wrapper around PDO.

 

Here's an example of a simple utilization of my DBWrapper:

 

try {
  $mydb = new DBWrapper();

  // get the data and do what you want with it...
  $data = $mydb->readData('SELECT * FROM users where ID = :id', $userid);
  ...

} catch (DBWrapperException $ex) {
  die($ex->getMessage());
}

 

The DBWrapper / readData() will take care of creating the PDO object, if it's not already created. It will take care of creating a proper PDOStatement inside it, use bindParam and protect against SQL injections, etc. etc. You can pass the parameter value as simple value to read and storeData() (doing so, DbWrapper will determine itself datatypes to map data to in DB), but you can also pass it an array mapping parameter value => datatype....

I'm not gonna write all of its functionality's here, but you can guys get the idea, I think.

Cheers. Pat

 

[Barand]

So while your writing

try {
  $mydb = new DBWrapper();

  // get the data and do what you want with it...
  $data = $mydb->readData('SELECT * FROM users where ID = :id', $userid);
  ...

} catch (DBWrapperException $ex) {
  die($ex->getMessage());
}

I write

$data = $mydb->prepare("SELECT * FROM users WHERE ID = :id");
$data->execute(['id'=>$userid]);

Not sure I'm totally sold on the benefits.

[PatRoy]

22 minutes ago, Barand said:

So while your writing

I write

$data = $mydb->prepare("SELECT * FROM users WHERE ID = :id");
$data->execute(['id'=>$userid]);

Not sure I'm totally sold on the benefits.

I think you write a bit more than that...

Your two-liner code should really be: 

try {
  $mydb = new PDO($dsn, $username, $password, $options);
  $data = $mydb->prepare("SELECT * FROM users WHERE ID = :id");
  $data->execute(['id'=>$userid]);
} 
catch (Exception $e) {
  die($e->getMessage());
}

Plus, I was under the impression that it's still better to also bindValue / bindParam, to also set its proper datatype, protecting against SQL Injections, which you are not using... 

With your way, I have to re-enter all of the DB configs everytime, which is the least of the problems for me, but still... 

My example was for a very basic usage. But what if you want to say do multi-inserts, AND commit on every inserts so that if one insert fails, the others still go through, and you can get a list of failed ones? I've had this case! And this class can handle this, without writing too much code.

I am not going to debate 'why' I think my class is useful here... Before saying anything, have a look at its UsageExamples.php and decide for yourself. If you don't think it to be useful, then by all means, don't use it  I'm not trying to sell it here , but rather just share.

 

 

Edited April 17, 2020 by PatRoy
edit

[Barand]

When I connect (once at the top of the script, not for every query) I set the option to throw exceptions so I don't have to check for them every time. This brings me back to the two lines.

[PatRoy]

2 hours ago, Barand said:

When I connect (once at the top of the script, not for every query) I set the option to throw exceptions so I don't have to check for them every time. This brings me back to the two lines.

It is still one line more than me, if you consider the fact that I do it the same way with my DBWrapper 😛 

But really, this discussion is non-sense and pointless to me, especially if one doesn't even look at the usage examples / documentation and doesn't care to see how it can really help down the line. Again, you take, or not... Whatever suits you, no bad feelings héhé.

Cheers. P.