As we prepare to his DST again, I sat in the open mic call this morning. A simple request to Lotus..
- Listen to Episode 28 with Scott of Lotus that not only got downloaded an amazing number of times, but had tons of info.
- Listen to Episode 27 with Andy and Rob of Technotics as we talk all about DST impact
But I noticed one thing from listening to the callers this morning. A lot of people have been doing upgrades, changes and deploying applications since the last DST time change. Yet everyone has the same question. What version has what fix and if I upgrade is it done?
So Lotus, we need a simple scenario listing in a whitepaper or technote that shows the outcome of where they are now and what steps are needed. Such as:
- you already patched for spring and have not changed the server code
- you patched for spring and have now upgraded to X.xx version
- you just installed version 7.0.2, is there any patches I need or is it included?
- you were in a version 6.5.x and patched in spring, we then upgraded to 6.5.6. Do I have to repatch?
- Other countries are now going into time changes, so if I have international servers/users I now need to patch those? (like Australia and Brazil)
- THE BIGGIE: We found that the .5 release of the agent had a 'bug' that the agent was processing faster than the document collection. So this means it possibly could miss some documents while getting others, in the same mailfile. This meant revisiting our heaviest calendar users again at the end of this week after the big changes on Apr 25th we performed.
- We had certain resources never take the time changes while others did in the same resources.nsf database
- We had appointments scattered across patched and unpatched machines like a great attempt at humor
- We had users parent documents for repeating meetings acting funny when the children documents fell into the new time changes
- Until the .7 release of the agent the script errors were causing us major headaches
- We had a site apply the OS patches with the Domino servers running and never had restarted them. Even through the DST patches we applied in Domino. So they suddenly were in Baja, California for all their meetings.
This didn't sneak up on anyone, we all just waited till the last minute to start worrying. Software vendors didn't roll patches until Nov 2006 for the OS and February 2007 for some applications. This put testing in a time crunch. This falls back to careful planning, testing and implementations.
Have no fear, we will all live through it, alibi a little late or early for meetings. So no harsh words for your administration team, I am sure they worked their butts off by the volume of emails and questions I received the past few weeks. Not to mention that the DST podcast with Scott Vrusho of Lotus is the 2nd largest download in the IdoNotes podcasts. (now don't mention it to Scott, he thought no one listened to it until he started getting emails and phone calls, LOL). So buy them a lunch instead, most worked late into the nights.
What this means is that while the agent completes, it might have skipped certain documents in the user's calendar. Of course, this is totally random. We found most mailfiles were good, but then some would have appointments that did and did not convert. Running the new agent again against these mailfiles seemed to solve the issue. WAS far as we can tell because there is not enough time to go through the properties of each entry and find the timezone values.
So good luck once again..
IBM is holding daily "Open Mic DST Calls". These calls are intended to
provide a forum for our customers to bring their questions, concerns etc..
around DST to us! Our goal is to provide them with the information they
need and to answer the questions that they have in order to ready their
systems and WPLC products for the DST changeover.
IBM has planned calls for Tuesday - Friday (March 6th - 9th) and March
12th from 12:00pm - 1:00pm Eastern.
Tuesday 3/6 -
Toll free: 1-888-732-6202
Participant Passcode: 893498
Toll free: 800 214 0745
Toll: +1 719 457 0700
Participant Passcode: 158121
Toll free: 1-888-373-5705
Participant Passcode: 547292
Toll free: 1-866-237-3252
Participant Passcode: 163964
Also, the demo videos can be found here:
New Videos show sample scenario of applying DST change to Notes and Domino
New video instructions (screen capture with audio narration) have been provided. These videos demonstrate how a Notes calendar is impacted by the DST change and show one scenario of applying the necessary updates to allow for the new Daylight Saving Time definitions. The download link to the videos is embedded within the "C&S Agents" technote below.
Title: Agents for updating Calendaring and Scheduling entries and Resource Reservation entries for Daylight Saving Time (DST) 2007
In addition, a video has been created to demonstrate how to use the Java Time Zone Update (JTZU) tool for updating DST information in your Java Runtime Environment(s). The JTZU video can be accessed via the following updated technote:
Title: Using the IBM Time Zone Update Utility for Java (JTZU) with Lotus software products
You then go in an edit and resave the document (or run an agent to refresh them all) and you get the following.
Ignore the Adminp statement if you edit and resave. It is the saving action that does it apparently.
But as a quick note, look for another new agent (18.104.22.168) to come out and fix some of the looping script errors we received on numerous servers while running the server based agent against the mailfiles. We saw this on more than a few customers across versions of templates as well as Domino versions. It drove us nuts, and wasted a lot of time to have to go into the text files and remove the offending user mailfile to get the agent to run on. Until it encountered another one and looped again. Now some ran without incident. Others stopped more than 20 times on larger sites.
I also talk about the order we did things and across the product lines.
5:30am - So first up for me was the Sametime servers. Others were prepping the DWA, calendar and RnR stuff. I have some of those to do in a bit, but I started with our internal servers first. Running the JTZU patch took far too long to search the Sametime systems. You really cannot run this tool in interactive mode since then you need to specify what gets updated and you have no clue. It even prompts you that letting it search could take hours. It really only took a few minutes when all was said and done to find what needed updating. It did take a while to run however.
6:16am - This was incredibly frustrating when the IBM support site was up and down all morning also. Yes we have knowledgebase locally, but it is faster to web grab some of the files. Also, it also would be nice not to get just random error messages on documents not existing when you know they do.
7:00am - First batch of RnR changes completed and one test mailfile set done. One weird error on one customer and the rest went smooth so far.
8:00am - Script errors when the calendar agent runs on a bad mailfile in the text list. We find endless script loops running. Removing the last mailfile attempted (and all previous completed ones) from the text list and restarting the agent fixes it. Some clients have no issues at all, others have a handful that cause grief. It has you going back to each server and making sure it it not looping.
8:30am - The path for managed and hosted server is an issue, so we created numerous agents with different drive letters that we can fire off. Now AS/400 and some random servers ever have different data paths from the norm. Standardization I say, standardization.
8:50am - Encounter first Domino Directory in foreign language. Script in agents only works on English views. It says it can't find the Server\Mail Users view. Which is there, however it is Servidor\Usuarios do Correio. First glance doesn't show where the agent grabs that view name to change it.
So I will give another (after much sleep) overview tomorrow on steps, commands and other things we figured out and streamlined as we went along to make your life easier.
1. To run as multiple instances (i.e., four instances), copy/paste the agent multiple times in the same database, and change the name to "AdminAgent1", "AdminAgent2" etc.
2. Ensure you have the server setup to run the desired number of concurrent agents in the Server document in the Domino Directory. The "Max concurrent agents" setting is found on the Server Tasks -> Agent Manager tab. Note: There are separate settings for "Daytime Parameters" and "Nighttime Parameters," make sure that you set each as desired.
3. Repeat steps 1-5 from the section above on configuring the agent to run in the background:
- in step 1, ensure that multiple TXT files are used to evenly divide the list of files to process
- in step 2 ensure the individual agents are edited to point to the individual TXT files
- in step 6, simply issue "Tell AMGR Run" for each of the individual copies of the agent:
i.e. for 4 agents it would be the following
Tell AMGR Run "mail\AgentDb.nsf" 'AdminAgent1'
Tell AMGR Run "mail\AgentDb.nsf" 'AdminAgent2'
Tell AMGR Run "mail\AgentDb.nsf" 'AdminAgent3'
Tell AMGR Run "mail\AgentDb.nsf" 'AdminAgent4'
So notice the key area about having enough Amgr threads defined to run all the instances you wish.
(4) is in relation to putting the OS patch on the clients. For one thing, Lotus suggests that the Notes client is closed when the patch is applied. How many of your users leave the client open all night long?
The computer should be restarted after the patch is applied and before restarting Notes. If the computer is not restarted after installing the patch, Notes will return the old time zone information for time zones other than the current time zone.
A lot is riding on this client restart or even patching. I am also not sure how you are forcing your users with machines at home or laptops to do it. If you start scheduling meetings across the clients that do and do not have the OS patch, then you will get variances in what they see and how correct they are.
Now to toss in some confusion, (5) should be done as soon as possible in relation to the OS updates. This is finely designed and choreographed dance folks. The amount of errors that DWA users will get relies heavily on this. See technote #1241063 for the alternative issues. From simple error pop-ups to meetings getting scheduled in Greenwich Mean Time, I think we have a problem here. There is no way to get all those DWA user machines.
(6) moves on to the RnR database and the users mailfiles. Now, if the users have local replicas also, they should be grabbing the change agent through replication before you run it on the server too. See technote #1254639. The RnR Manager must be shut down when the agent runs, and you even get a prompt while running this from the database Action menu. Note, you will be signing these agents with some id file that needs at least editor access to the RnR database.
I will have more info for you to follow this one as we finalize and formulate the plan to update servers globally, hopefully it helps.
Has anyone noticed that the "patched" systems also incorrectly "fix" the past?
At least on a Windows system (Notes 6.x client). If your client and server have been patched, try creating (or updating) a Calendar entry for, say, November 1, 2006. You'll notice the time is incorrectly set to Daylight Saving time, which was not the case in 2006.
I don't know if this is a bug with Microsoft's patch, or IBM's implementation in Notes/Domino, but this can create havoc on applications with historical data.
Once again, this is a local Microsoft issue since it can only read one time zone at a time, not Lotus Notes. But be warned...