Re: Is translating server messages really worth it?

Hi,

I know that $SUBJECT is a bit curious from someone who's been translating
PostgreSQL server messages for quite some time now (I checked, I've been
doing it for 20 years...). But still, I'm wondering.

Most of the time when I'm at a customer's office trying to help them with
their PostgreSQL cluster, I tell them to set lc_messages to C, so that
server messages are in english. It's kind of weird or funny to do the
translation, and tell people not to use it. But, yeah, I don't know if it's
a good thing or not, but when someone tries to search if someone else
already met some weird server messages, this someone has more chances to
get a hit with an english message than with a french message (though I
guess it's the same for other non-english languages). If you have to use a
tool such as pgbadger, this kind of tool only knows English messages.

So, yeah, I'm kinda wondering if it makes sense to translate server
messages. For client tools, such as psql or pg_dump or vacuumdb, it
definitely makes sense. But the server logs? I pretty much don't know. It's
a lot of work, with some nearly-impossible-to-translate messages.

Anyway, I was wondering how you feel about this.

And I have another question, quite a bit related :) If a file (let's say
psql-fr.po) is not translated at 80%, it's not distributed. But I was
wondering if it was only this file (psql-fr.po) or all the files for this
language? I'm considering leaving the postgres-fr.po file without any
translation, but keep the other files up to date.

Thank you for your comments.

Regards.

--
Guillaume.

On Mon, Sep 25, 2023 at 12:46 PM Guillaume Lelarge
<guillaume(at)lelarge(dot)info> wrote:
> And I have another question, quite a bit related :) If a file (let's say psql-fr.po) is not translated at 80%, it's not distributed. But I was wondering if it was only this file (psql-fr.po) or all the files for this language? I'm considering leaving the postgres-fr.po file without any translation, but keep the other files up to date.

The 80% rule always struck me as odd. It might make sense if falling
under the threshold meant that the translation was classified as a
second tier translation, or whatever. But it's a binary pass/fail
condition -- there are no reasonable gradations. A 79% complete
translation that nobody really has access to is 100% useless.

Intuitively, the rule seems like it almost has it backwards; I would
imagine that even a fairly incomplete translation could be almost as
useful as a complete translation, mostly for the reasons that you went
into already. Such a translation would still have to be kept up to
date for those messages that really matter -- those things tend to
change at least as often as anything else. Plus the general quality of
the translations seems crucial. Quality over quantity.

--
Peter Geoghegan

Hey everbody!

>Hi,
>
>I know that $SUBJECT is a bit curious from someone who's been
>translating PostgreSQL server messages for quite some time now (I
>checked, I've been doing it for 20 years...). But still, I'm wondering.
>
>Most of the time when I'm at a customer's office trying to help them
>with their PostgreSQL cluster, I tell them to set lc_messages to C, so
>that server messages are in english. It's kind of weird or funny to do
>the translation, and tell people not to use it. But, yeah, I don't know
>if it's a good thing or not, but when someone tries to search if
>someone else already met some weird server messages, this someone has
>more chances to get a hit with an english message than with a french
>message (though I guess it's the same for other non-english languages).
>If you have to use a tool such as pgbadger, this kind of tool only
>knows English messages.
>
>So, yeah, I'm kinda wondering if it makes sense to translate server
>messages. For client tools, such as psql or pg_dump or vacuumdb, it
>definitely makes sense. But the server logs? I pretty much don't know.
>It's a lot of work, with some nearly-impossible-to-translate messages.

Thanks for a great question! I answer it whenever I give an i18n talk or
describe the i18n process.

We are all biased as consultants. We know how to solve problems
efficiently. We know where to search for solutions (usually the source
code). That's why we set C-style messages, run our standard procedures,
follow our methods, apply our own monitoring, etc. That's just a part of
our professional workflow, but it doesn't mean people shouldn't use
localized messages.

Maintaining a glossary
Want it or not, many areas use localized terms and words in everyday
life, e.g., education, medicine, military, etc. We might lose a lot if
we translate only the user interface part, omitting server messages.

Satisfying requirements
It sounds bureaucratic; nevertheless, we need a fully translated product
to pass some government, military, and education requirements. I've met
a situation when a product lost tender only because it wasn't localized.
And I don't need to remind which products are always localized and in
good shape: mssql, oracle, etc.

Feedback
You said already that some messages are nearly impossible to translate.
However, that mainly indicates the low quality of a source message, not
the absence of linguistic tools to express that message. Giving feedback
to developers increases source code quality.

Error codes
If only every single error message had a non-localized error code, we
wouldn't have this question at all. You agree that searching for "E0042"
is much easier than copy-pasting part of an error message. Even more,
having an error code gives you the freedom to change, adapt, and improve
error messages. It's not a rare story when messages are pretty different
between major server versions. We have SQLSTATE error codes, but AFAIR,
they're not mandatory for logging and only apply to SQL errors.

>
>
>Anyway, I was wondering how you feel about this.
>
>And I have another question, quite a bit related :) If a file (let's
>say psql-fr.po) is not translated at 80%, it's not distributed. But I
>was wondering if it was only this file (psql-fr.po) or all the files
>for this language? I'm considering leaving the postgres-fr.po file
>without any translation, but keep the other files up to date.
>
>Thank you for your comments.
>
>Regards.
>
>
>--
>Guillaume.

> On 26 Sep 2023, at 00:48, Peter Geoghegan <pg(at)bowt(dot)ie> wrote:

> there are no reasonable gradations. A 79% complete
> translation that nobody really has access to is 100% useless.

Those 75% or 79% or whatever could also very well cover nearly 100% of the
common cases, which would increase the value proposition of the translated
part. It's of course impossible to say, but it doesn't seem unlikely.

--
Daniel Gustafsson

Hi,

Le mar. 26 sept. 2023 à 11:28, Pavlo Golub <pavlo(dot)golub(at)gmail(dot)com> a écrit :

> Hey everbody!
>
> Hi,
>
> I know that $SUBJECT is a bit curious from someone who's been translating
> PostgreSQL server messages for quite some time now (I checked, I've been
> doing it for 20 years...). But still, I'm wondering.
>
> Most of the time when I'm at a customer's office trying to help them with
> their PostgreSQL cluster, I tell them to set lc_messages to C, so that
> server messages are in english. It's kind of weird or funny to do the
> translation, and tell people not to use it. But, yeah, I don't know if it's
> a good thing or not, but when someone tries to search if someone else
> already met some weird server messages, this someone has more chances to
> get a hit with an english message than with a french message (though I
> guess it's the same for other non-english languages). If you have to use a
> tool such as pgbadger, this kind of tool only knows English messages.
>
> So, yeah, I'm kinda wondering if it makes sense to translate server
> messages. For client tools, such as psql or pg_dump or vacuumdb, it
> definitely makes sense. But the server logs? I pretty much don't know. It's
> a lot of work, with some nearly-impossible-to-translate messages.
>
>
> Thanks for a great question! I answer it whenever I give an i18n talk or
> describe the i18n process.
>
> We are all biased as consultants. We know how to solve problems
> efficiently. We know where to search for solutions (usually the source
> code). That's why we set C-style messages, run our standard procedures,
> follow our methods, apply our own monitoring, etc. That's just a part of
> our professional workflow, but it doesn't mean people shouldn't use
> localized messages.
>
> Maintaining a glossary
> Want it or not, many areas use localized terms and words in everyday life,
> e.g., education, medicine, military, etc. We might lose a lot if we
> translate only the user interface part, omitting server messages.
>
> Satisfying requirements
> It sounds bureaucratic; nevertheless, we need a fully translated product
> to pass some government, military, and education requirements. I've met a
> situation when a product lost tender only because it wasn't localized. And
> I don't need to remind which products are always localized and in good
> shape: mssql, oracle, etc.
>
>
I don't believe we have this issue in France. At the very least, I've never
heard about this in France. Having a french manual, yes. But French
translation of server logs, I don't think so.

> Feedback
> You said already that some messages are nearly impossible to translate.
> However, that mainly indicates the low quality of a source message, not the
> absence of linguistic tools to express that message. Giving feedback to
> developers increases source code quality.
>
>
Definitely true. To be honest, I don't do this for the .po files, but I
frequently report typos or parts I don't understand in the manual.

> Error codes
> If only every single error message had a non-localized error code, we
> wouldn't have this question at all. You agree that searching for "E0042" is
> much easier than copy-pasting part of an error message. Even more, having
> an error code gives you the freedom to change, adapt, and improve error
> messages. It's not a rare story when messages are pretty different between
> major server versions. We have SQLSTATE error codes, but AFAIR, they're not
> mandatory for logging and only apply to SQL errors.
>
>
Totally agree, though I don't think it will happen at all. It would also
help a lot coding tool such as pgBadger.

Thanks for your comments.

>
>
> Anyway, I was wondering how you feel about this.
>
> And I have another question, quite a bit related :) If a file (let's say
> psql-fr.po) is not translated at 80%, it's not distributed. But I was
> wondering if it was only this file (psql-fr.po) or all the files for this
> language? I'm considering leaving the postgres-fr.po file without any
> translation, but keep the other files up to date.
>
> Thank you for your comments.
>
> Regards.
>
>
> --
> Guillaume.
>
>

--
Guillaume.

> On 26 Sep 2023, at 22:17, Guillaume Lelarge <guillaume(at)lelarge(dot)info> wrote:
> Le mar. 26 sept. 2023 à 11:28, Pavlo Golub <pavlo(dot)golub(at)gmail(dot)com <mailto:pavlo(dot)golub(at)gmail(dot)com>> a écrit :

> Satisfying requirements
> It sounds bureaucratic; nevertheless, we need a fully translated product to pass some government, military, and education requirements. I've met a situation when a product lost tender only because it wasn't localized. And I don't need to remind which products are always localized and in good shape: mssql, oracle, etc.
>
> I don't believe we have this issue in France. At the very least, I've never heard about this in France. Having a french manual, yes. But French translation of server logs, I don't think so.

Interesting, I think we have the opposite. While not mandatory in Sweden I
believe "software available in Swedish" is a quite common constraint in
gov/council etc tenders (and unless there is a constraint on which messages it
applies to, it applies to all of the software but not the manual).

--
Daniel Gustafsson

On 25.09.23 23:48, Peter Geoghegan wrote:
> The 80% rule always struck me as odd. It might make sense if falling
> under the threshold meant that the translation was classified as a
> second tier translation, or whatever. But it's a binary pass/fail
> condition -- there are no reasonable gradations. A 79% complete
> translation that nobody really has access to is 100% useless.

At FOSDEM 2023 [0], I discussed the 80% rule with some other
translations projects (e.g., KDE), but no one else seems to have such a
system, so maybe it wasn't the right idea.

[0]: https://archive.fosdem.org/2023/schedule/track/translations/

One specific example of a thing I would like to avoid is that the --help
output has a wild mix of translated and untranslated lines.

Of course, this could be avoided by having the whole --help output as
one translation catalog entry, which is exactly what we have not been
doing, because that makes it harder to update between versions.

So I don't know ...

On 25.09.23 20:45, Guillaume Lelarge wrote:
> So, yeah, I'm kinda wondering if it makes sense to translate server
> messages. For client tools, such as psql or pg_dump or vacuumdb, it
> definitely makes sense. But the server logs? I pretty much don't know.
> It's a lot of work, with some nearly-impossible-to-translate messages.
>
> Anyway, I was wondering how you feel about this.

I think it would be good if we could prioritize messages somehow. I
think there'd be a lot of value with, like, the most important 500
messages in the server.

I have no idea how to arrange that.

> And I have another question, quite a bit related :) If a file (let's say
> psql-fr.po) is not translated at 80%, it's not distributed. But I was
> wondering if it was only this file (psql-fr.po) or all the files for
> this language? I'm considering leaving the postgres-fr.po file without
> any translation, but keep the other files up to date.

For the record, this is per file.

> On 27 Sep 2023, at 17:28, Peter Eisentraut <peter(at)eisentraut(dot)org> wrote:

> One specific example of a thing I would like to avoid is that the --help output has a wild mix of translated and untranslated lines.

We clearly want to avoid that.

> Of course, this could be avoided by having the whole --help output as one translation catalog entry, which is exactly what we have not been doing, because that makes it harder to update between versions.

Could we group messages that have all-or-nothing translations required, like
--help output, with separate msgctxt contexts? So "pg_dump --help" would be
"msgctxt pg_dump_help_screen" in the po file, which then could be analyzed in
order to make the call for including translationfiles or not?

--
Daniel Gustafsson

Le mer. 27 sept. 2023 à 17:28, Peter Eisentraut <peter(at)eisentraut(dot)org> a
écrit :

> On 25.09.23 23:48, Peter Geoghegan wrote:
> > The 80% rule always struck me as odd. It might make sense if falling
> > under the threshold meant that the translation was classified as a
> > second tier translation, or whatever. But it's a binary pass/fail
> > condition -- there are no reasonable gradations. A 79% complete
> > translation that nobody really has access to is 100% useless.
>
> At FOSDEM 2023 [0], I discussed the 80% rule with some other
> translations projects (e.g., KDE), but no one else seems to have such a
> system, so maybe it wasn't the right idea.
>
> [0]: https://archive.fosdem.org/2023/schedule/track/translations/
>
> One specific example of a thing I would like to avoid is that the --help
> output has a wild mix of translated and untranslated lines.
>
> Of course, this could be avoided by having the whole --help output as
> one translation catalog entry, which is exactly what we have not been
> doing, because that makes it harder to update between versions.
>
> So I don't know ...
>
>
I do like the idea of publishing a translation only if there's enough
translated strings. That's the 80% rule. Of course, the "80" part is kind
of arbitrary but you need a number and 80 is probably as good/bad as
anything else (at least above 60).

--
Guillaume.

Le mer. 27 sept. 2023 à 17:32, Peter Eisentraut <peter(at)eisentraut(dot)org> a
écrit :

> On 25.09.23 20:45, Guillaume Lelarge wrote:
> > So, yeah, I'm kinda wondering if it makes sense to translate server
> > messages. For client tools, such as psql or pg_dump or vacuumdb, it
> > definitely makes sense. But the server logs? I pretty much don't know.
> > It's a lot of work, with some nearly-impossible-to-translate messages.
> >
> > Anyway, I was wondering how you feel about this.
>
> I think it would be good if we could prioritize messages somehow. I
> think there'd be a lot of value with, like, the most important 500
> messages in the server.
>
> I have no idea how to arrange that.
>
>
Sounds interesting but no idea either.

> > And I have another question, quite a bit related :) If a file (let's say
> > psql-fr.po) is not translated at 80%, it's not distributed. But I was
> > wondering if it was only this file (psql-fr.po) or all the files for
> > this language? I'm considering leaving the postgres-fr.po file without
> > any translation, but keep the other files up to date.
>
> For the record, this is per file.
>
>
OK, thanks.

--
Guillaume.

On 2023-Sep-28, Daniel Gustafsson wrote:

> Could we group messages that have all-or-nothing translations required, like
> --help output, with separate msgctxt contexts? So "pg_dump --help" would be
> "msgctxt pg_dump_help_screen" in the po file, which then could be analyzed in
> order to make the call for including translationfiles or not?

This is an interesting idea. So for that kind of thing (help screens)
we would require a very high percentage in order to include it with the
bundle (say 90%), but it would be a very small catalog so easy to
translate; and for other things, say the bulk of server messages, we
would require a lesser percentage in order to be included (even 60%
could be more than good enough IMO).

I'm not sure how that would work, however.

--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/

> On 4 Oct 2023, at 11:10, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org> wrote:
>
> On 2023-Sep-28, Daniel Gustafsson wrote:
>
>> Could we group messages that have all-or-nothing translations required, like
>> --help output, with separate msgctxt contexts? So "pg_dump --help" would be
>> "msgctxt pg_dump_help_screen" in the po file, which then could be analyzed in
>> order to make the call for including translationfiles or not?
>
> This is an interesting idea. So for that kind of thing (help screens)
> we would require a very high percentage in order to include it with the
> bundle (say 90%), but it would be a very small catalog so easy to
> translate; and for other things, say the bulk of server messages, we
> would require a lesser percentage in order to be included (even 60%
> could be more than good enough IMO).
>
> I'm not sure how that would work, however.

Neither do I, I need to educate myself more on the translation infra and
process in order to have a better idea if it's at all feasible. I am
interested in making it easier to contribute a new translation, or reboot a
stale one, and lowering the overall requirement while keeping the requirement
high on the most relevant bits seem like a way forward.

--
Daniel Gustafsson

Daniel Gustafsson <daniel(at)yesql(dot)se> writes:
> On 4 Oct 2023, at 11:10, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org> wrote:
>> ... I'm not sure how that would work, however.

> Neither do I, I need to educate myself more on the translation infra and
> process in order to have a better idea if it's at all feasible. I am
> interested in making it easier to contribute a new translation, or reboot a
> stale one, and lowering the overall requirement while keeping the requirement
> high on the most relevant bits seem like a way forward.

Drive-by comment: one way to reduce the workload is to be more
aggressive about questioning which messages need to be translated
at all. We have mechanisms for indicating which server messages
are worthy of translation (use errmsg_internal, or just use elog,
when it's not), but few people think hard about when to use them.
(I sure don't.) I'm not sure if there's a corresponding mechanism
for our client programs, but probably there should be.

Maybe it'd be worth making a sweep to find messages that we don't
really think the average user will ever see, and marking those
as not to be translated. Then going forward, we'd have to have
a policy that people should think about that when writing new
errors.

I'm not sure how this would play into the upthread point about
regulatory compliance. If there's somebody out there who has a hard
requirement for 100% translation, I'm afraid we will never satisfy
that. But a deliberate policy of only translating messages we think
are likely to show up in normal use might fall foul of policies that
are okay with our current level of effort.

regards, tom lane

> On 11 Oct 2023, at 05:12, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Maybe it'd be worth making a sweep to find messages that we don't
> really think the average user will ever see, and marking those
> as not to be translated. Then going forward, we'd have to have
> a policy that people should think about that when writing new
> errors.

I think that would be a good idea regardless. Lowering the barrier to entry
for a new translation without lowering the quality would make it less daunting
to start a new language, or revitalize a stale one.

We already have a style guide for messages which is followed, adding to it a
policy for internal/translatable messages seems like a low cognitive overhead.
The documentation already have these:

errmsg_internal(const char *msg, ...) is the same as errmsg, except that
the message string will not be translated nor included in the
internationalization message dictionary. This should be used for “cannot
happen” cases that are probably not worth expending translation effort on.

errdetail_internal(const char *msg, ...) is the same as errdetail, except
that the message string will not be translated nor included in the
internationalization message dictionary. This should be used for detail
messages that are not worth expending translation effort on, for instance
because they are too technical to be useful to most users.

Adding section on translated/internal to the style guide might be a good idea.

> I'm not sure how this would play into the upthread point about
> regulatory compliance. If there's somebody out there who has a hard
> requirement for 100% translation, I'm afraid we will never satisfy
> that.

There is that. I think optimizing for making life easier for translators by
allowing them focus on the messages that matter and are expected to be seen is
more important.

--
Daniel Gustafsson