Andrew Gregory's Web Pages
View from Snake Hill, Lake Ballard, 29°26'40"S 120°36'19"E
A microcontroller based NTP server, synchronized to a GPS receiver.
PLEASE NOTE: The CCS port of the Microchip TCP/IP stack is copyright by CCS and not allowed to be re-distributed. If you want a copy, you can download it from CCS, provided you have download rights. You may need to purchase a maintenance agreement from CCS. Requests or discussions to me for copies of the CCS TCP/IP stack will be ignored.
Assuming my dynamic DNS, home Internet link, home web server, and the NTP server itself are all up and functioning, the following is the current live status web page of my NTP server. You'll need an SVG capable browser such as Opera, Firefox or a very recent Safari. Older Safaris and Internet Explorer require the Adobe SVG viewer.
In the middle of 2007 I upgraded my Linux server box from a 486DX2/66 to a 1GHz Athlon. That old box had been sharing my Internet connection, files, a dot matrix printer, emails, and acting as a time synchronization server.
As well as upgrading to a faster box, I also upgraded from a simple ADSL modem to a modem-router, ditched the printer, shifted mail-handling responsibilities back to my ISP and also intended shifting the file sharing to a dedicated hardware NAS device, but that last item ran into some technical difficulties.
That left just the time synchronization, and while I could have just duplicated the old setup, I thought it would be nice to have a bit of dedicated hardware for that, too. There were some other incentives too. I'd recently got into microcontroller programming with Microchip PICs, and a future project was going to need Internet enabling. I felt I should get some experience before tackling that project and a NTP server seemed the ideal way.
This design isn't particularly accurate, but it's generally within a second or two of UTC. If you're interested in something more accurate, and/or don't or can't build something like this yourself, you can purchase similar off-the-shelf devices. For example, TimeTools.
I started with Microchip's ENC28J60 Ethernet Interface chip. This is a fabulous chip, available in through-hole packaging making it easy for part-time electronics folks like me to handle. Then I upgraded my CCS C compiler to their version 4 and grabbed their TCP/IP library.
The next step was selecting a PIC micro. I initially selected an 18LF2620 and prototyped with that, however, it turned out that I needed a couple more I/O pins than were available, so I switched to an 18LF4620. My prototype has found a second life as a UPS monitor.
The board is intended for re-use with a future project, so there are some extras that the NTP server doesn't use, such as the four external Flash memory sockets around the left of the micro, and the 14-pin expansion header. Below the micro is a DS1305 real-time clock (and 1F super cap for backup power). That isn't strictly required either, but it allows the time server to keep time without the GPS receiver, and will be required in the future project.
My first prototype used a 7805 linear regulator, but since the power supply is powering the GPS receiver as well as the server, it got a bit too hot (90°C) even with a heatsink. The switch-mode power supply runs at a more modest 52°C.
The front of the server is a green power LED and two red/green bi-colour LEDs for status. The middle status blinks every time an NMEA GPRMC message is received - green when the location data is valid, and red when it isn't. The right status currently just blinks when the server broadcasts an NTP time packet.
The back of the server has (left to right):
The interface to the GPS receiver is via the serial port. I originally used 4-way cable which carried power, ground, receive and transmit signals. There was no wire left to carry the pulse-per-second signal. Subsequently, I've changed that for a 5-way cable and brought the pulse-per-second to the server.
My initial design did not support the pulse-per-second signal. During development I recognized that by the time my server had processed the NMEA data and the RMC message, a reasonably fixed amount of time may have passed since the time as reported in the message. To handle that I incorporated an adjustable offset. Trial-and-found found the best offset to be about two seconds. I had expected the serial data to align more closely with the correct time, so it was a bit surprising. At the time I thought any attempt to synchronize with the pulse-per-second may not have turned out very well, but it has proved to be quite good.
In the end, I'm happy that I can revert to using any GPS receiver with a serial port, without requiring access to the pulse-per-second. However, in the middle of February 2008, some months after my initial implementation, I decided to bring the pulse-per-second out of the GPS. That required some internal modifications in my receiver and a new cable with an extra conductor for the signal. Then I had to modify my server board to direct the pulse-per-second to an interrupt-capable pin.
Documentation on the NTP protocol is readily and freely available. In particular there is the web site of David Mills, the original designer of NTP: Network Time Synchronization Research Project. In particular, the Reference and Implementation Guide PDF is highly recommended reading.
The requirements for my server are very simple. It only needs to synchronize with the GPS, and does not need to synchronize with any other NTP server. Much of the complications of NTP involve NTP clients synchronizing with NTP servers (which includes lower-stratum NTP servers synchronizing with higher-stratum servers). Without that complexity, all my server needs to do is transmit the current time as it sees it.
That means that I only had to deal with the first 48-bytes of any NTP UDP packet received. Any other bytes are ignored and discarded. The replies back to the client are the same set of 48-bytes, modified appropriately.
NTP Time Formats, when representing a date, is the number of seconds from 00:00:00 1 January 1900 UTC.
| NTP Time Formats | |
|---|---|
| NTP Short | 32 bit: 16 bit whole seconds, 16 bit fractional seconds |
| NTP Long | 64 bit: 32 bit whole seconds, 32 bit fractional seconds |
Packet structure:
| Byte Offset | Bit Offset | |||||
|---|---|---|---|---|---|---|
| 0 | 2 | 5 | 8 | 16 | 24 | |
| 0 | LI (2 bit) | VN (3 bit) | Mode (3 bit) | Stratum (8 bit) | Poll (8 bit) | Precision (8 bit) |
| 4 | Root Delay (32 bit, NTP short format) roundtrip delay to the reference clock | |||||
| 8 | Root Dispersion (32 bit, NTP short format) dispersion (variance) to the reference clock | |||||
| 12 | Reference Identifier (32 bit) | |||||
| 16 | Reference Timestamp (64 bit, NTP long format) | |||||
| 24 | Originate Timestamp (64 bit, NTP long format) | |||||
| 32 | Receive Timestamp (64 bit, NTP long format) | |||||
| 40 | Transmit Timestamp (64 bit, NTP long format) | |||||
Packet fields:
| Field | Description |
|---|---|
| LI | Leap Indicator, 2 bit unsigned integer:
|
| VN | Version, 3 bit unsigned integer, eg 4 |
| Mode | 3 bit unsigned integer:
|
| Stratum | 8 bit unsigned integer:
|
| Poll | 8 bit signed integer representing the maximum time between successive messages in log2 seconds. Typical poll values are between 6 and 10 (64 and 1024 seconds). |
| Precision | 8 bit signed integer representing the precision of the system clock in log2 seconds. For example, -18 corresponds to about 1 microsecond. |
The NTP server will be sent a UDP packet formatted as above. My server only processes the Mode and Transmit Timestamp fields.
To reply, my server makes the following changes:
The resulting 48-byte packet is sent back to the client, and the server resumes waiting for the next request.
The first thing was that the Ethernet and TCP/IP stack takes up 32KiB by itself. However, that includes an HTTP server and DHCP client. This is a serious amount of firmware space, and limits you to the top end of Microchips offerings, since you need at minimum a 48KiB part, but most likely a 64KiB part.
The CCS TCP/IP stack (it's a port of Microchip's v3.75 stack) is easy to use. However, CCS have introduced a bug that blocks retries on TCP connections, resulting in dropped connections after only a few minutes (at the first error). I undid their change, but long-duration connections still stall, but at least they recover after a second or two and will keep going for hours. The delay, though, makes my Telnet Com Port implementation less useful. I also had to tweak their ICMP module to support the timestamp packet.
By far the largest changes I made to the TCP/IP stack involved the HTTP server. In order to support the "HTTP Time Protocol", HEAD requests and date/time stamps must be supported. So I implemented those. Next I needed to implement basic authentication, to stop casual changing of settings. Finally I tweaked it to output the "no-cache" header where required. I also slipped in some preliminary code to support retrieving pages from external Flash memory, but haven't used it, yet.
The CCS compiler itself is quite good, even though the new v4 is suffering from quite a few code generation bugs. Compared to the IAR MSP430 compiler, it makes interfacing to both internal and external peripherals very easy.
One of the most buggy features is the new "accessmod" system, that allows variables to be allocated in storage other than the internal RAM. That could include the internal EEPROM, an external EEPROM, or whatever, really. The intention of the feature is to allow you to declare your variables with a single keyword modifier that tells the compiler which memory to use to store it, and the compiler will use your provided read/write functions to access the memory. The compiler handles all the allocation/layout and addressing. The problem is it often generates incorrect calls, especially for writing. Plus the code it generates is huge. I started out using accessmod to interface to the internal EEPROM and external RAM in the DS1305, but when I replaced the compiler code with my own that manually managed the allocation and access, not only did all the read/write related bugs disappear, but ~2KiB dropped off the firmware size too!
When I originally bought my receiver, back in 2000, it performed quite well. Since then, it's degraded noticably. It can now take several hours to get a position lock, and readily loses lock. Being on a window sill (it faces south), it doesn't have the greatest view of the sky, so that could be part of the reason. Another part can be seen by examining the satellite data on the status page. Comparing against my Globalsat BT-338 (which uses a Sirf 3 chipset, versus Sirf 1) I almost always see satellites out of position, extra satellites where there shouldn't be any, and missing satellites where there should be some. I'm wondering if there isn't a bug in the firmware, and/or something has slightly changed in the GPS communications that the old firmware can't handle.
I had my PCBs made by Futurlec. Their prices are excellent, as is their board quality. I've only gone for their double-sided, soldermasked, and silkscreened options so far (they look so nice!), but they do offer cheaper boards without the soldermasks and silkscreening. However, for both of my orders so far I've had to wait a month from placing the order to getting the boards.
In order to use Futurlec, you need to use Protel to do the PCB. Luckily, I have access to Protel. If you don't have access to Protel, I found the software from ExpressPCB to be very easy to use, although you are then forced to manufacture with ExpressPCB. The only major thing missing was verification that the PCB design matched the source schematic. Apart from that, it did everything I wanted. I never used their PCB manufacturing service. The only comment I can make, is that it looks at least twice as expensive as the Futurlec prices, which to be fair, is probably due to the incredibly fast turnaround times ExpressPCB promise.
Maybe it's just my computer, but I've found the Internet Time function of Windows XP to be quite unreliable. It's sometimes very difficult to get it to sync with my server. In comparison, Time Tools NTP Server Monitor was able to regularly poll my server without any problems.
To add extra time servers to the Internet Time list, you need to go to the
registry key HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\DateTime\Servers
and add your new time server FQDN
or IP address (just give them the next
numbers in sequence as their Names). You can set the Data of the Default value
to one of your newly added servers.
To change the update interval from the default of once per week, go to the
registry key HKML\SYSTEM\CurrentControlSet\Services\W32Time\TimeProviders\NtpClient
and change the Data of the SpecialPollInterval. The value is in
seconds. The default (one week) interval is 1 week = 604800 seconds. A 1 day
interval is 86400 seconds.
Coding on microprocessors is often quite challenging - especially trying to get all the functionality you want into the available memory. All the web pages in my server are stored in the internal program EEPROM, and making the code as small as possible was very important.
However, just as important, was to ensure all the code emitted was valid. Since I was using tables to lay out configuration forms, that implied the use of HTML 4.01 and the corresponding long DOCTYPE. Still, I was able to arrange the code so that the DOCTYPE only appears once in the program. For the rest of the HTML I studied the DTD and found that quite a bit can be trimmed off while staying valid.
In the middle of February 2008 I modified the system to utilize the pulse-per-second from my GPS receiver. That has improved the precision, but not to the extent that I'd hoped. The earlier code relying solely on the NMEA data would frequently be 1-1.5 seconds out (not to mention relying on a large fudge factor to even get close), while the new code is now typically less than a second, sometimes within 30ms, but more typically about 300ms out (compared against time.nist.gov, time.windows.com and several servers out of *.au.pool.ntp.org) - all without any fudge factors at all! However, I'm unable to explain why it isn't even more accurate than it currently seems to be.
My GPS outputs GGA, GSA, and RMC messages every second. Every five seconds it adds the GSV messages to the set. The problem is that all that data is more than it can send in one second at 4800 baud. Sadly, the baud rate is fixed so I can't adjust it to 9600. It's made worse by the fact that my GPS always outputs the RMC message last. The result is that sometimes the RMC time stamp associated with a pulse isn't processed until after the next pulse! I handle that by treating a second RMC message received within a pulse period as the true message.
Some people have asked about the cost of all this. All up, it has cost me quite a bit, as I bought many of the more unusual parts at low volumes through rather expensive electronic component suppliers. Keeping to low volumes, but using more economical (but less convenient) suppliers, the total parts cost of one unit comes to about AUD$120. About one third of that is just the PCB. An external plugpack and cables are additional.
This would allow "net time \\servername /set /yes" to
work. The packet captures look relatively simple. However, they don't appear to
be the usual TCP/UDP packets that the CCS TCP/IP stack supports. It looks like
I'd have to do some serious modifications to that stack.
Lots of acronyms there! In short, this would allow SNMP management and monitoring of the server. However, the MIB is not due for review until May 2008 (or thereabouts), and it could be longer before it is ready to be implemented.
This seems to be the standard Linux way of sharing GPS data. The protocol looks fairly simple, but I'd have to add code to parse the GGA and GSA messages.
It would be really nice to be able to support firmware upgrading through the web interface. However, that would require enough external storage for a temporary copy of the firmware, plus the extra processing code to handle the upload, checking, and bootloader to reprogram the PIC. This would require at least one, probably two, external 64KB EEPROMs.
I currently have about 1500 bytes of program space available. That probably means one of SMB or gpsd, but not both. It would also count out SNMP support and firmware updating entirely. Microchip have just one 40-pin through-hole PIC available with more than 64KB of program space - the 96KB 18LF4685.
VALID»
Last updated: 2008-03-27. Copyright © 1999-2008.
http://www.scss.com.au/family/andrew/gps/ntp/
