Subject: Re: Predictable Internet Time

From: Joe Touch <[email-masked]>
Date: Mon, 27 Mar 2017 12:34:10 -0700

Hi, all,

I've submitted the time frame discussion intended to resolve this issue,
which also recently arouse on another mailing list. Further discussion
on this draft will occur on the ART mailing list ([email-masked]).

Joe

-----------

A new version of I-D, draft-touch-time-01.txt
has been successfully submitted by Joe Touch and posted to the
IETF repository.

Name:		draft-touch-time
Revision:	01
Title:		Resolving Multiple Time Scales in the Internet
Document date:	2017-03-27
Group:		Individual Submission
Pages:		17
URL:            https://www.ietf.org/internet-drafts/draft-touch-time-01.txt
Status:         https://datatracker.ietf.org/doc/draft-touch-time/
Htmlized:       https://tools.ietf.org/html/draft-touch-time-01
Htmlized:       https://datatracker.ietf.org/doc/html/draft-touch-time-01
Diff:           https://www.ietf.org/rfcdiff?url2=draft-touch-time-01

Abstract:
   Internet systems use a variety of time scales, which can complicate
   time comparisons and calculations. This document explains these
   various ways of indicating time and explains how they can be used
   together safely. This document is intended as a companion to
   Internet time as discussed in RFC 3339.

---

From: "Patrik Fältström" <[email-masked]>
Date: Tue, 28 Mar 2017 08:37:56 +0200

Joe,

I have read your I-D and like it! Let me start there :-)

What I think is not clear enough is the problem with POSIX, and it should be more clear in some place, maybe section 6.1, that POSIX definition which is in use in quite a number of systems do not handle leap second very well. Too many do believe the time_t definitions include the number of seconds since epoch when in reality it does not (as you note in the definitions).

One could even question whether it is Continuous as two seconds will have the same number since epoch around the addition of a leap second (the last second of the day with leap second and the first the day after the addition of leap second). I claim it is not as if you look at also fractions of seconds it will go backward like:

:
(N-1).0
(N-1).1
:
(N-1).9
N.0
N.1
:
N.9
N.0
N.1
:
N.9
(N+1).0
(N+1).1
:

There are some people that have suggested a change, for example <http://www.cl.cam.ac.uk/~mgk25/time/c/> but I have not seen any movement. Maybe you know more than me on this?

   Patrik

On 27 Mar 2017, at 21:34, Joe Touch wrote:

> Hi, all,
>
> I've submitted the time frame discussion intended to resolve this issue, which also recently arouse on another mailing list. Further discussion on this draft will occur on the ART mailing list ([email-masked]).
>
> Joe
>
> -----------
>
> has been successfully submitted by Joe Touch and posted to the IETF repository.
>
> Name:		draft-touch-time
> Revision:	01
> URL:            https://www.ietf.org/internet-drafts/draft-touch-time-01.txt Status:         https://datatracker.ietf.org/doc/draft-touch-time/
> Htmlized:       https://tools.ietf.org/html/draft-touch-time-01 Htmlized:       https://datatracker.ietf.org/doc/html/draft-touch-time-01 Diff:           https://www.ietf.org/rfcdiff?url2=draft-touch-time-01
>
> Abstract:
>    Internet systems use a variety of time scales, which can complicate   time comparisons and calculations. This document explains these   various ways of indicating time and explains how they can be used   together safely. This document is intended as a companion to   Internet time as discussed in RFC 3339.

---

From: Nico Williams <[email-masked]>
Date: Tue, 28 Mar 2017 12:39:18 -0500

On Tue, Jan 03, 2017 at 06:35:03PM +0000, Stewart Bryant wrote:
> Yes, the system should use a leap-second-free constant-duration-seconds time
> for everything. It is only humans that need the variable jumpy version of
> time presented to them, and that is a UI issue.

I don't think that's quite right.

Unix time is canonically a count of seconds since an epoch and admits no
leap seconds.  A conversion of Unix time to adjusted (i.e., with leap
seconds added) time since the same epoch would be nice, as well as:

 - conversion from either to broken down time
    - with seconds count up to 60 when converting from seconds-with-leaps
 - formatting of dates from either as well as from broken down time
    - but when converting from seconds-with-leaps the formatted string
      can show seconds-in-minute values of 60
    - whereas when converting from Unix seconds the formatted string can
      never show seconds-in-minute values of 60
 - parsing to broken down time and to seconds-since-epoch types
    - ignoring leap seconds when converting to Unix time
    - correctly accounting for all leap seconds prior to that time when
      converting to seconds-with-leaps-since-epoch

It's all about data typing.  In POSIX C API terms, we need new functions
to deal with seconds-with-leaps-since-epoch, and obviously a lot of
prayer because there's no way in C to distinguish one integer type from
the other (well, the new type could be wrapped in a struct..).

We will have to deal with multiple kinds of time as long as different
applications/users need different definitions of time.  Astronomers may
need corrected time, while bankers may want to ignore leap seconds --
all on the same systems (same OSes, same code libraries).

Functionally this is about data typing -- preferably strong data typing.

Smearing is a third time representation that also needs conversion to
the other types (more opportunity for bugs!); it leaves a bad taste.  It
may work for some apps though, but all nodes had better smear in exactly
the same way -- within your tolerances -- if your app depends on time
for synchronization.

Nico
--

---

From: Joe Touch <[email-masked]>
Date: Tue, 28 Mar 2017 10:47:30 -0700

Hi, Patrik,

> What I think is not clear enough is the problem with POSIX, and it should be more clear in some place, maybe section 6.1, that POSIX definition which is in use in quite a number of systems do not h…

I can make that more clear, but AFAICT POSIX time is *defined* as
seconds since the Unix epoch *not counting* leap seconds at all.

> One could even question whether it is Continuous as two seconds will have the same number since epoch around the addition of a leap second 
It is Continuous by definition.

When UTC adds a leap second, nothing different happens to POSIX time.

...
> There are some people that have suggested a change, for example <http://www.cl.cam.ac.uk/~mgk25/time/c/> but I have not seen any movement. Maybe you know more than me on this?
AFAICT, that's just attempting to redefine the <time.c> interface as
returning UTC rather than POSIX time.

Which works *IF* your machine has access to updated leap-second
information, e.g., via NTP.

Joe

---

From: Michael Breuer <[email-masked]>
Date: Tue, 28 Mar 2017 21:09:44 +0200

Hej Joe

On 03/28/17 19:47, Joe Touch wrote:
>> What I think is not clear enough is the problem with POSIX, and it
>> should be more clear in some place, maybe section 6.1, that POSIX
>> definition which is in use in quite a number of systems do not
>> handle leap second very well. Too many do believe the time_t
>> definitions include the number of seconds since epoch when in
>> reality it does not (as you note in the definitions).
> 
>> One could even question whether it is Continuous as two seconds
>> will have the same number since epoch around the addition of a leap
>> second

That's a quite complex topic to find precise wording. If the last
statement would be right, wouldn't this be a contradiction to your
earlier statement, POSIX time be defined *not counting* leap seconds?
AFAIK POSIX time is continuous and not counting leap seconds. So when
UTC adds a leap second, something quite different happens to POSIX time.
It will change the leap-second-offset between UTC and Unix time.

To my opinion, any time system utilising leap seconds isn't continuos at
all. There are continuous time systems like TAI or GPS system time (or
any other satellite navigation system time) and POSIX time, which (by
design) shift away from UTC over time.

Michael

> ...
>> There are some people that have suggested a change, for example
>> <http://www.cl.cam.ac.uk/~mgk25/time/c/> but I have not seen any
>> movement. Maybe you know more than me on this?
>>> I've submitted the time frame discussion intended to resolve this
>>> issue, which also recently arouse on another mailing list.
>>> Further discussion on this draft will occur on the ART mailing
>>> list ([email-masked]).
>>> 
>>> Joe
>>> 
>>> -----------
>>> 
>>> A new version of I-D, draft-touch-time-01.txt has been
>>> successfully submitted by Joe Touch and posted to the IETF
>>> repository.
>>> 
>>> Name:		draft-touch-time Revision:	01 Title:		Resolving Multiple
>>> Time Scales in the Internet Document date:	2017-03-27 Group:
>>> Individual Submission Pages:		17 URL:
>>> https://www.ietf.org/internet-drafts/draft-touch-time-01.txt
>>> Status:
>>> https://datatracker.ietf.org/doc/draft-touch-time/ Htmlized:
>>> https://tools.ietf.org/html/draft-touch-time-01 Htmlized:
>>> https://datatracker.ietf.org/doc/html/draft-touch-time-01 Diff:
>>> https://www.ietf.org/rfcdiff?url2=draft-touch-time-01
>>> 
>>> Abstract: Internet systems use a variety of time scales, which
>>> can complicate   time comparisons and calculations. This document
>>> explains these   various ways of indicating time and explains how
>>> they can be used   together safely. This document is intended as
>>> a companion to   Internet time as discussed in RFC 3339.
> 
>

---

From: Nico Williams <[email-masked]>
Date: Tue, 28 Mar 2017 14:16:55 -0500

On Sat, Dec 31, 2016 at 04:32:20PM -0500, Phillip Hallam-Baker wrote:
> Well the astronomers are at it again. They are messing about with time
> which is a terrible idea. Specifically they are adding another leap second.
> 
> [...]
> 
> So to remove the confusion entirely, while preventing the need for a
> discontinuous adjustment of the drift between UTC and TAI, I propose the
> Predictable Internet Time (PIT) as follows.

We have two kinds of time today: UTC on the one hand (has leaps) and
TAI (and POSIX) on the other (has no leaps).  Converting between the two
requires a leap second history.

Leap seconds are not that predictable.  Significant seismic events
(e.g., the 2004 earthquake and tsunami) can measurably alter the speed
of Earth's rotation on its axis.

You propose a new kind of time that lies between UTC and TAI.
Conversions between this new type and TAI would still need adjustment as
leap second rates change, while conversions to UTC would still need a
leap second history _and_ said adjustments.

I imagine many would code-inject PIT at run-time as being POSIX time.

> Nor is changing the definition of UTC to effect simultaneous change a
> feasible solution because to do so would be to demand the astronomers
> accept a diminution in their own prestige.

Or a new time "data type" just for them, but leaving UTC = TAI + 36
after a certain date because UTC has been used widely for non-astronomy-
related purposes.

> With a suitable definition, PIT could create a condition in which it would
> only take a decision by one major government to force a change on the
> astronomers. The commercial advantages of PIT over UTC are obvious - fewer
> things are going to break for no good reason. That is an argument that
> every politician is willing to listen to.

Do we have this much influence?  If we do, why not just... lobby for
fixing UTC and creating an TAC (tempt astronomique coordiné) to serve,
for astronomers, the function that UTC serves today.

> While a variance of a second or three between New York and London might be
> inconvenient, [...]

For some apps that could be terrible.  Now developers will have to think
about 200% more time conversions.  Not exactly thrilling.

Nico
--

---

From: Joe Touch <[email-masked]>
Date: Tue, 28 Mar 2017 12:18:37 -0700

> earlier statement, POSIX time be defined *not counting* leap seconds?

POSIX ignores leap seconds - whether added (typical) or removed (if that
ever happens).

> UTC adds a leap second, something quite different happens to POSIX time.
POSIX doesn't know or care - it just keeps ticking.

> It will change the leap-second-offset between UTC and Unix time.

That happens because of something UTC did, not something POSIX did.

> any other satellite navigation system time) 
GPS reports TAI+19s, but at least one other satellite system jumps with
leaps (GLONASS).

> and POSIX time, which (by
TAI does too.

TAI and POSIX also drift away from each other in unpredictable ways
because TAI uses SI seconds and POSIX has no precise definition (it
appears to assume a second derived from a mean solar day, but it isn't
precisly spec'd).

Joe

---

From: Tony Finch <[email-masked]>
Date: Tue, 28 Mar 2017 21:24:01 +0100

> When UTC adds a leap second, nothing different happens to POSIX time.

Clearly not. Either it jumps backwards at the start of the leap second, or
at the end of the leap second, or it stops for a second, or it runs slow
for some  period of time in a controlled manner (e.g. leap smear) or an
uncontrolled manner (NTP swinging around in a wild effort to resync).

The POSIX formula that specifies the translation from UTC to time_t
implies that it jumps back at the end of the leap second.
http://pubs.opengroup.org/onlinepubs/9699919799.2016edition/basedefs/V1_chap04.html#tag_04_16

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/  -  I xn--zr8h punycode
South German Bight, Humber, Thames: Southwesterly 4 or 5, increasing 6 at
times. Slight or moderate. Occasional rain, fog patches. Moderate or good,
occasionally very poor.

---

From: Joe Touch <[email-masked]>
Date: Tue, 28 Mar 2017 13:28:03 -0700

> uncontrolled manner (NTP swinging around in a wild effort to resync).

POSIX time just keeps ticking.

It is only in relation to UTC that you would notice anything. And it
isn't POSIX that is jumping - it is UTC.

> http://pubs.opengroup.org/onlinepubs/9699919799.2016edition/basedefs/V1_chap04.html#tag_04_16
That equation is clearly introduced as "A value that approximates..."

Leap seconds and the difference between 86400 seconds/day and SI
definitions are where the approximation errs.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 28 Mar 2017 14:36:54 -0700

Hi, all,

Please take this discussion to ART ([email-masked])

Joe

---

From: "Patrik Fältström" <[email-masked]>
Date: Wed, 29 Mar 2017 07:52:15 +0200

On 28 Mar 2017, at 19:47, Joe Touch wrote:

> I can make that more clear, but AFAICT POSIX time is *defined* as seconds since the Unix epoch *not counting* leap seconds at all.

Well, one of the problems is that the specification of POSIX time is a bit unclear.

> When UTC adds a leap second, nothing different happens to POSIX time.

No, that is not correct if you look at the day when the leap second is added.

POSIX is number of (days * 86400) plus seconds inside the current day. This implies if there is a day with a leap second there will be 86401 seconds which also will be the first second the following day.

> AFAICT, that's just attempting to redefine the <time.c> interface as returning UTC rather than POSIX time.

The UNIX time_t struct do deliver UTC, including leap seconds, but the number of seconds from epoch is wrong regarding leap seconds. Includes one leap second within the day of the leap second being added and without leap seconds otherwise.

> Which works *IF* your machine has access to updated leap-second information, e.g., via NTP.

Correct, if one have access to leap seconds, one could adjust the number of seconds and that could happen already today as POSIX definition is a bit unclear on this. But people have not implemented it. So instead for example Google smear the leap second over the day so that there is not bump in the road, and they run NTP servers that do give back this smeared time that differs from UTC.

All just because a. unclear information from POSIX and b. bad implementations in *ux.

   paf

---

From: Joe Touch <[email-masked]>
Date: Wed, 29 Mar 2017 07:43:50 -0700

Hi, Patrik,

On 3/28/2017 10:52 PM, Patrik Fältström wrote:
> POSIX is number of (days * 86400) plus seconds inside the current day.

POSIX asserts that a "day" = 86400 * "seconds" but defines neither day
nor second.

It definitely says it does not include leap seconds, so there cannot be
"plus seconds inside the current day" - doing that assumes that POSIX
days are correlated to UTC days, but that's never stated.

>  This implies if there is a day with a leap second there will be 86401 seconds which also will be the first second the following day.

A UTC day with a leap second clearly has to use a different definition
of either 'second' or 'day' than POSIX, which requires that every day
has exactly 86400 seconds.

> The UNIX time_t struct do deliver UTC, including leap seconds, but the number of seconds from epoch is wrong regarding leap seconds. Includes one leap second within the day of the leap second being…
time_t in POSIX is supposed to return a signed integer of the number of
seconds since the Unix epoch.

any more complicated structure is an attempt by a system to convert its
internal time to UTC, which requires knowing about leap seconds as well
as knowing the relation of the local internal "second" vs the SI version.

> Correct, if one have access to leap seconds, one could adjust the number of seconds and that could happen already today as POSIX definition is a bit unclear on this. 

IMO, it's very clear that leap adjustments are not part of POSIX time.

> But people have not implemented it. So instead for example Google smear the leap second over the day so that there is not bump in the road, and they run NTP servers that do give back this smeared time that differs from UTC.
They're not using POSIX time then; they're using an internal
approximation of UTC to provide time to NTP.

Joe

---

From: Nico Williams <[email-masked]>
Date: Wed, 29 Mar 2017 10:41:03 -0500

On Wed, Jan 04, 2017 at 09:21:29AM +0000, Stewart Bryant wrote:
> On 03/01/2017 19:54, Cyrus Daboo wrote:
> >>This is true, but any specific local time will always occur at a
> >>specific universal time, so this isn't actually a problem.
> >
> >No! That is not true because a future local time is not guaranteed to
> >be at a specific universal time because local government can change
> >their time zone (or more likely) their daylight savings time
> >transition at any time between now and said future time. i.e., you
> >cannot "pre-cache" all the local time -> UTC mappings for a future
> >recurring event and then just use the UTC values, because the time
> >zone definition may change.
> 
> So an application sensitive to that needs to specify its timing in
> terms of local time, but that should not force the fundamental time
> distribution system to operate in that mode.

Sure it does!  If you want to interchange calendaring events (which...
people _do_) then those events must refer to local time, and they must
be stored in local time form.  Otherwise the same issues Cyrus mentions
crop up.

This is hardly problematic.  It just means that

a) every system needs to be able to convert to/from local time (but that
   was _already_ the case even when you believed that local time was
   only for interaction with a user) from using any and all timezones
   (which was not already the case for you, but it's fine anyways),

b) (a) implies that tzdist is needed and TZ data has to be kept up to
   date.

> Maybe we need two time distribution systems, one that accurately does
> the ranging and flywheel management and distributes precision time for
> technical purposes, and a separate one that manages the offsets and
> jumps relative to the underlying precision continuous time
> distribution system.

Yes, of course.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Wed, 29 Mar 2017 10:45:37 -0500

On Tue, Jan 03, 2017 at 03:06:57PM -0500, Phillip Hallam-Baker wrote:
> +1

Indeed.

> Case in point, the time displayed on my Honda Oddy does not correct for
> daylight savings correctly because some pimple brain decided to vary the
> date of the switchover and there is no way to update the firmware without
> spending stupid amounts of money to do so.
> 
> This is the sort of thing I would like to be able to avoid in IoT. The
> problem being that to do so requires vendors to think about problems as
> something other than an excuse to sell a 'pro' version or updates.

Trouble is, this is one more protocol for IoTs to speak.  One more
source of those vulnerabilities that make IoTs IoSs.

> Not being able to predict with accuracy when a future time expressed in
> civil time will occur is a problem.

It's easy to predict that actual changes will be withing +/- 1 hour
though.  For events scheduled far into the future this is mostly a
non-issue.

I'm not interested in more complaining about politicians making our life
hell.  Fact is they get to and they do and they have and they will keep
doing it.  So we should arrange our affairs as to minimize the pain and
just move on.

Nico
--