Thu Nov 13 00:17:34 PST 2014
All systems are up and available. The scheduled maintenance on Nov 12 is complete.
Wed Nov 12 15:04:04 PST 2014
All systems are up and available.
Conferencing via well.com and user.well.com will be unavailable from 10pm-midnight US Pacific Time on Wednesday, Nov 12. We apologize for the inconvenience.
Tue Nov 06 18:30:30 PDT 2014
Investigating system outage on well.com/user.well.com
Tue Oct 28 13:13:23 PDT 2014
All systems are up and available.
SECURITY UPDATE
Sometime on Wednesday, Oct 29, we will update the SSL certificate associated with iris.well.com (the WELL's email server, aka Zimbra, aka Iris) to one that uses the SHA-256 (aka SHA-2) hashing algorithm. The old SHA-1 algorithm has been deprecated, and many sources, including our certificate provider, are urging that the old certificates be replaced as soon as possible. (SSL certificates are used by browsers to ensure that the site you are accessing is the site that it claims to be.)
Note that this change could have incompatibility impacts on older browsers and email software. Please see welltech.374.920 for further details.
Sun Oct 19 02:41:13 PDT 2014
All systems are up and available.
At about 11:30pm on Oct 18, there was a configuration glitch that temporarily caused email to be unavailable due to apparent password problems. The glitch was fixed and service restored at about 2:30am. If you tried to change your password during this period, you may find that you now have two different passwords - one for mail, and one for everything else. We recommend updating your password once more to get them back in synch.
Fri Oct 10 10:15:24 PDT 2014
On Monday Oct 13 (Columbus Day) at 9:00 am PDT, the server at well.com (aka user.well.com) will be shut down for maintenance by Amazon Web Services. The maintenance is scheduled to last two hours, though we expect it will be much shorter than that.
During the maintenance period, the UNIX and web interfaces to our conferences, Picospan and Engaged, will not be available. The UNIX shells as well as the Alpine mail program will also be unavailable. However, email will still be available at iris.well.com. Both sending and receiving email will also continue to be available via POP and IMAP, as usual. The www.well.com website will also remain up.
We apologize for any inconvenience this maintenance presents.
Wed Sep 10 17:06:24 PDT 2014
A major update to our web conferencing software, commonly referred to as "Engaged," has been completed. To report any new bugs or to read the scope of the updates please see the System Status Reports topic in News or WellTech.
Some highlights:
For more, including updates to parsing and formatting, please see the System Status Reports topic in News or WellTech. Questions and bug reports are welcome.
All systems remain up and available.
Tue Apr 15 08:06:24 PDT 2014
All systems are up and available.
Tue Apr 8 13:19:51 PDT 2014
The email system received an urgent security update today. It is possible that your browser or mail program will complain about not being able to verify the identity of iris.well.com. Please see welltech.374.334- for info.
Tue Oct 1 14:54:53 PDT 2013
Planned Outage: Email Maintenance and Upgrade Rescheduled for 10/2 - 10/3
03:08 POP access is now available: This completes the Zimbra 8 upgrade: Please see the welltech conference for information about the update, and to report any issues. 03:00 IMAP and Alpine access is now available: you can send and read mail via your IMAP client or the Apline command-line interface. Things will continue to be slow for a while. 02:50 SMTP service is available: you can send mail via your IMAP or POP client, and newly arrived email is being delivered to your mailboxes. Forwarding should also be working at this time. 02:41 Web access to your email is available: you can send and read mail via your browser. Things will be slow for a couple of hours, but they should be working. 01:54 Upgrade complete, starting system
System will be started one component at a time, and may take some time to become completely available.23:51 Zimbra 8 installed, now configuring. 21:54 Installing Zimbra 8. 21:54 Preparing system for Zimbra 8 update. 20:00 Initial setup for upgrade is underway.
There will be no outbound well.com email service -- and no access to inbound mail -- starting Wednesday 10/2 at 8pm US Pacific Time. while we upgrade the email system to the latest version, Zimbra 8. We expect to restore services at 4am Pacific Time, Thursday, 10/2.
Incoming mail will safely queue on the server until we are back. During the outage, you will not be able to send, receive, or view mail, nor can it forward.
We apologize for any inconvenience. For updates, please watch this page. For discussion, go to the System Status discussions in the News or WELLtech conferences.
Mon Sep 30 9:20:39 PDT 2013
Access has been restored. The upgrade will be rescheduled, due to circumstances beyond our control.
Log in and see the System Status topic in the News or WELLtech conferences if you want to get the details or ask questions.
Mail was queued and then delivered after the system came back up. Sorry for the inconvenience.
Sun Sep 29 22:52:39 PDT 2013
Update: Access will be restored ASAP, due to circumstances beyond our control.
This may be before or after the original target time of 4am. We do not have a new estimated time yet. Log in and see the System Status topic in the News or WELLtech conferences for additional details.
We apologize for any inconvenience.
Sun Sep 29 20:31:22 PDT 2013
Planned Outage: Email Maintenance and Upgrade 9/29 - 9/30
Initiial setup for upgrade is underway. Expect an update to this page at about 10:30pm US Pacific Time.
There will be no outbound well.com email service -- and no access to inbound mail -- starting Sunday 9/29 at 8pm US Pacific Time. while we upgrade the email system to the latest version, Zimbra 8. We expect to restore services at 4am Pacific Time, Monday, 9/30.
Incoming mail will safely queue on the server until we are back. During the outage, you will not be able to send, receive, or view mail, nor can it forward.
We apologize for any inconvenience. For updates, please watch this page. For discussion, go to the System Status discussions in the News or WELLtech conferences.
Tue Apr 9 22:14:46 PST 2013
All systems are running normally.
____
Fri Apr 5 13:09:45 PDT 2013
There will be a brief outage of well.com (the shell/picospan host), user.well.com (web conferencing/Engaged), and www.well.com (the web server) Tuesday at 10pm US Pacific Time (note that this is a rescheduled date). Each system needs to be rebooted so we can enable controls to prevent disk overruns. All systems should be back and running by 10:30pm.
____
Tue Feb 26 22:54:49 PST 2013
All systems are running normally.
____
Tue Feb 26 12:26:10 PST 2013
Mail is currently unavailable as we clean up from last week's upgrade. Mail is expected to return by midnight tonight. See you then!
____
Thu Feb 21 06:09:16 PST 2013
Last night's mail system upgrade appears to have been successful. However, it took 4 hours longer than we scheduled for due to unforeseen issues with the Amazon "Cloud". See welltech.370 for details of what's new.
____
Wed Feb 20 20:00:00 PDT 2013
Mail is currently unavailable as a software upgrade is performed. Mail is expected to be available by 2am Thursday morning. See you then!
Update as of 02:47:00: we're running way behind schedule. Currently, we're expecting to be back online at around 4:30am. We apologize for the inconvenience, and especially the longer than expected outage.
____
Mon Jan 7 14:48:22 PST 2013
Due to an NFS issue, user WEB directories and short-term backup recovery snapshots are unavilable on well.com. User web pages are still available at http://www.well.com, and files can be updated with scp or sftp at www.well.com, but files cannot be viewed or updated from the well.com host.
In addition, you may see problems when trying to login or logout from the Web-based conferencing software (Engaged). To login, go to http://www.well.com. The error on logout can be ignored.
____
Mon Dec 3 16:32:55 PST 2012
A system reboot is scheduled for Tuesday, Dec 4, 11pm US Pacific Time. The reboot will last about 5 minutes, and will effect shell and conferencing access (both via SSH and the Web).
____
Wed Nov 21 22:31:45 PST 2012
The restart of our SMTP server is taking longer than anticipated. We have an open ticket with Amazon Web Services to investigate why the system is not shutting down properly. We don't yet have an ETA.
____
Wed Nov 21 17:10:13 PST 2012
Amazon has alerted us that our SMTP server is running on "degraded hardware". To get things back to normal, we will be rebooting the server later tonight (around 10pm). If all goes well, it should only take about 5 minutes.
Alpine users are most likely to see any effects - while the server is rebooting, Alpine will complain if you try to send email. Just resend the mail a few minutes later if this happens to you. Other customers may seem some minor delays on mail being received or sent, but in almost all cases, the delayed mail should be delivered properly some time after the system comes back up.
Sorry for any inconvenience this reboot will cause...
____
Tue Oct 2 22:03:27 PDT 2012
All systems are up and running.
However, we have changed our DNS authoritative server, and some DNS servers are being slow to pick up this change. If you are having trouble connecting to any WELL machines, you may want to change your DNS server on your router, Mac, or PC to use 8.8.8.8 as the primary server. However, be careful to keep a record of the previous settings, since you will likely want to change these settings back at some point. Your best bet is to wait until your normal DNS server picks up the updates, which should happen within a few hours in most cases.
NOTE: It looks like there have been a few cases of mail bouncing from about 7:40pm-8:50pm, and again from about 10:05pm-10:20pm US Pacific Time. Please ask senders to resend if you are missing mail from this period.
We apologize for any inconvenience resulting from this necessary update.
____
Fri Jun 15 12:18:22 PDT 2012
Amazon EC2 services, the cloud where our servers reside, has called for us to shut down and relaunch well.com and user.well.com, the servers where WELL conferencing lives. Tomorrow, Saturday June 16, those services will be unavailable from approximately 9pm until 11:00pm Pacific Daylight Time while wolfy migrates services to new instances. (Last night there was significant disruption that was probably unrelated to this next action, but which also came from problems at the Amazon cloud. Sorry for the inconvenience.) Thank you for your patience and loyalty.
____
Fri Jun 15 09:38:22 PDT 2012
All systems are up and running.
____
Fri Jun 15 01:22:04 PDT 2012
Email is down due to problems in the Amazon Cloud. We do not have an ETA at this time, but email may not be available until late morning or later.
____
Wed Mar 14 15:04:28 PDT 2012
We will be rebooting well.com and user.well.com Wednesday evening (Mar 14) at about 11pm US Pacific Time to make a minor configuration change. The reboot and maintenance will take about 20 minutes, and will effect access to the conferences via both Picospan and the Web, as well as all shell services (including alpine mail). We apologize for the inconvenience.
____
Thu Mar 1 22:26:11 PST 2012
(ETA changed to midnight - we're running a bit behind schedule)
On Thursday evening between about 9pm and midnight US Pacific Time, WELL conferencing via UNIX shell and the web will be down, with all access to well.com and user.well.com disabled for the duration. During this maintenance, we will be switching over to a new system kernel - we expect that this will put an end to the kernel panics we have seen over the last two months.
Access to your email via https://iris.well.com/, POP, and IMAP will not be affected by this maintenance. However, please note that access to email via the "alpine" mail program will not be available since well.com itself will be unavailable.
We apologize for the inconvenience of this maintenance, but random system crashes are even more inconvenient.
____
Thu Feb 23 21:15:19 PST 2012
ENGAGED IS DOWN - the database was accidentally trashed. We hope to have it back up by 10:30pm US Pacific Time.
____
Wed Jan 11 17:00:35 PST 2012
We did not reboot Tuesday, so we will be rebooting well.com tonight, Jan 11, at about 10pm US Pacific Time. The reboot will take 5 to 10 minutes, and will effect all uses of the conferences.
____
Wed Jan 11 09:37:59 PST 2012
All systems are up and running.
____
Tue Jan 10 12:19:25 PST 2012
We will be rebooting well.com tonight at about 10pm US Pacific Time in an effort to address the repeated system panics. The reboot will take 5 to 10 minutes, and will effect all uses of the conferences.
____
Fri Dec 16 15:43:25 PST 2011
After a reboot, people are back posting. All systems are up and running.
____
Fri Dec 16 14:55:25 PST 2011
We are having a problem with the well.com and user.well.com machines. While we resolve this problem, the conferences -- our forums -- are unreachable. Email at iris.well.com and the www.well.com web page remain up. We are sorry for the inconvenience.
____
Fri Dec 9 10:53:25 PST 2011
All systems are up and running.
____
Thu Dec 8 2011 16:57:24 PST 2011
Our service provider, Amazon, needs to reboot our systems to perform some sort of system update (presumably patches). Fortunately, we can prevent the automatic reboot by rebooting manually. The best time for us to do this is tonight, even though that means a short notice outage.
So, tonight at about 10pm US Pacific Time, each of the various well.com systems will be brought down then rebooted. Each reboot should take less than 5 minutes, though the reboot of the Zimbra machine may take a few minutes longer (it takes several minutes just for Zimbra to shut down, and another few minutes to start up).
We apologize for the short notice for this announcement.
____
Thu Oct 27 14:10:50 PDT 2011
Most remaining bugs appear to have been eradicated. If you encounter any, please report them in the cloud discussion in the welltech conference. Log in, shortcut is "g welltech" to get there.
If you use the PicoSpan interface and you're getting error messages on login, the problem may be configuration in your personal log in files. If you send a message to helpdesk, we can back up and replace your current log in files with basic files so that you can test each line to find the culprit.
____
Wed Oct 26 07:45:03 PDT 2011
The WELL is now back up and running in the Amazon Elastic Computing Cloud (EC2). Please report any bugs by going to the cloud discussion in the welltech conference. Log in, shortcut is "g welltech" to get there. If there is a malformed conference mentioned in an error message on your your Conference List page when you first connect to open a page at "https://user.well.com/engaged.cgi?" you may want to remove the bad conference temporarily by editing your Settings. Reporting a bug could win you one of a bunch of interesting books, so don't hesitate.
____
Mon Oct 24 16:42:39 PDT 2011
Migration down-time planned
On Tuesday night, Oct 25, we finish up our migration into the Amazon Elastic Computing Cloud (EC2). The last machines to be moved will be the main machine (well.com), the conferencing web server (user.well.com), and the web page server (www.well.com).
As part of this migration, conferencing (via both the command line and web browser), as well as all access to The WELL's web pages, will be disabled. Email will remain available, except for the "alpine" mailer. We're anticipating that the migration should be complete in about 6 hours: from 9pm PDT Tuesday evening to 3am PDT Wednesday morning.
Following the migration, you should be able to use conferencing and all web services as before, with only minor changes that most people will not see. If by some chance you do experience problems after the system is restored to service, don't try to change your settings please. If we are still sorting out the move that could make things harder to diagnose. Please just contact helpdesk with any bug reports or post in the System Status topic in the WELL tech or News conference, so a proper diagnosis can be made.
We apologize for any inconvenience this downtime may cause.
____
Tues Aug 30 13:07:22 PDT 2011
Performance of the email system remains sluggish - we are working to restore good service.
We apologize for any inconvenience this slowness may cause.____
Fri Aug 26 13:43:56 PDT 2011
On Monday night, Aug 29, we begin our migration into the Amazon Elastic Computing Cloud (EC2). The first machines to be moved will be the Zimbra mail server (iris.well.com) and a companion server.
As part of this migration, email will be disabled while the migration is under way. We're anticipating that the migration should be complete in about 6 hours: from 9pm PDT Monday evening to 3am PDT Tuesday morning.
During the migration, incoming email from other sites should be queued at those sites, and re-sent later after we begin accepting incoming email again. WELL customers will not be able to look at previously received email during the outage unless you have already moved that email onto your computer's storage device.
Following the migration, you should be able to continue using email using the settings that have worked for the last few weeks - nothing should change. If you encounter problems after the migration, please avoid changing your mail settings and report the problems to Helpdesk or post in System Status Reports.
We apologize for any inconvenience this downtime may cause.____
Thu Aug 18 21:37:03 PDT 2011
The WELL is now back up and running. Engaged is still presenting some problems, but for now, you can make your own repairs using the instructions on your Conference List page.
____
Thu Aug 18 14:54:03 PDT 2011
RAID failure knocked out some services, currently being restored
Around 1:00 am on the morning of Thursday, August 18th, home directories became unreachable at The WELL, causing all services except email to fail.
At 9:30 the following morning we we were able to place the following unadorned message on the well.com site:
Most functions of The WELL are currently not available due to a hardware failure of the RAID where most of our data is kept. We are working on restoring full service, but do not yet have an estimated completion time. We apologize for this inconvenience.
Please note that email via IMAP, POP, and the Web should be available at this time.
During that time we went through some replacement hardware options, settled on one, and got it installed at our co-location facility. At about 14:45 PDT logins became possible again.
RAIDs are designed to prevent such problems when a disk fails, and we are in the process of moving well.com to the cloud, so this failure was perplexing and hopefully very soon all questions will become moot. Some services are still being repaired and adjusted at this time. Comments are welcome, please email Helpdesk at the address on the footer of this page, or swing by the welltech conference.
We are sorry for the disruption.
____
Mon Aug 1 16:42:55 PDT 2011
Mail upgrade complete: If you can't get mail you need to change settings
All systems are up and running. We now have IMAP available for all mail customers. Please be advised that we have shut off all access to two older mail servers as of this afternoon as part of the upgrade to email started in 2009. If you are pointing to those servers in your settings, your mail will no longer work.
If you are not getting or sending email, you probably need to update some settings. You no longer have access to smtp.well.com nor to mh.well.com (mail.well.com). All incoming and outgoing mail operations will now involve the iris.well.com server. To get the appropriate Ports, look up your appropriate settings.
1. You should have access to your new email via a direct web browser login at iris.well.com now and henceforth. This is the easy, quick way for most people to see mail.
2. New mail help for all kinds of programs is now up at http://www.well.com/emailhelp.html with an overview of the system, general settings and a link to a page with more settings for specific mobile devices and computer email programs.
3. You can move your old email into the new environment by use of the utilities page if you wish.
4. There is a way to use Alpine for UNIX access to your mail. See the help page first.
5. We attempted to migrate you with settings matching your former preferences. In some situations we were not able to do that! In particular, if you were forwarding elsewhere and leaving a copy you are probably set up with straight forwarding now, and you will want to adjust that with a web login to iris.well.com.
If you have questions, contact helpdesk or ask fellow users who have solved the problem in the welltech conference. Thank you for your patience.
____
Mon Aug 1 09:59:46 PDT 2011
Mail upgrade is in progress.
All remaining mailboxes at The WELL are being transferred to the Zimbra email system at iris.well.com over the next several hours today. Please see the mail help page for instructions on setting up once your mailbox is converted, or use your web browser to go to iris.well.com. (Note that you cannot log in there until the upgrade script converts your mail settings.) Thank you for your patience.
____
Wed Jun 22 12:55:09 PDT 2011
All systems are up and running.
____
Monday, June 20, 2011 05:36:27 PM PDT
The Zimbra email system on iris.well.com will be unavailable Tuesday evening (Jun 21) between the hours of 10pm and midnight so that we can perform a required software update. The legacy email system will remain up during this period. Sorry for the inconvenience.
____
Fri May 13 13:50:40 PDT 2011
Members using the Legacy email server are finding their connections time out when they are trying to access mail from mail.well.com. We are working on the situation.