Incident - History.
System health incidents details and history (last 20).
September 24, 2024 (Resolved)
Delayed loading times for WebApps
For about 5 minutes around 2.40PM about 10% of the webapps had delayed loading times when called. The issue has been resolved.
INC0053
Delayed loading times for WebApps
For about 5 minutes around 2.40PM about 10% of the webapps had delayed loading times when called. The issue has been resolved.
INC0053
August 28, 2024 (Resolved)
RESOLVED - High utilization for WebApps stopped, under further observation
In the period from 1.09 AM to 1.18 AM, there were significantly longer loading times for some of the WebApps. The fault has been rectified and we are investigating the case further.
INC0052
RESOLVED - High utilization for WebApps stopped, under further observation
In the period from 1.09 AM to 1.18 AM, there were significantly longer loading times for some of the WebApps. The fault has been rectified and we are investigating the case further.
INC0052
August 28, 2024 (Investigating)
We are currently experiencing particularly high utilization of WebApp applications.
We are currently investigating a particularly high utilization of WebApp applications. Our engineering team is already working on this.
INC0051
We are currently experiencing particularly high utilization of WebApp applications.
We are currently investigating a particularly high utilization of WebApp applications. Our engineering team is already working on this.
INC0051
August 05, 2024 (Resolved)
RESOLVED - Server API issue (INC0049) is resolved
All services should now be functioning normally - servers itself where not affected in any time.
INC0050
RESOLVED - Server API issue (INC0049) is resolved
All services should now be functioning normally - servers itself where not affected in any time.
INC0050
August 05, 2024 (Investigating)
API isses for server management and creation
Our Engineering team is investigating an issue with processing of events in all regions. During this time users may experience failed events on servers.
INC0049
API isses for server management and creation
Our Engineering team is investigating an issue with processing of events in all regions. During this time users may experience failed events on servers.
INC0049
August 04, 2024 (Maintenance)
Performance update led to short-term unavailability (WebApps)
Due to performance updates, parts of the WebApp systems were unavailable for approx. 1 minute. All systems are fully available again.
INC0048
Performance update led to short-term unavailability (WebApps)
Due to performance updates, parts of the WebApp systems were unavailable for approx. 1 minute. All systems are fully available again.
INC0048
July 26, 2024 (Resolved)
[RESOLVED] Delayed login to Cloud Mails and Cloud Management Suite
On July 26, between 6 and 7 a.m. (AM / CEST), we experienced problems logging in to various services. These have now been completely resolved.
INC0047
[RESOLVED] Delayed login to Cloud Mails and Cloud Management Suite
On July 26, between 6 and 7 a.m. (AM / CEST), we experienced problems logging in to various services. These have now been completely resolved.
INC0047
July 19, 2024 (Identified)
Information on the disruption of Windows systems in connection with Crowdstrike
We are currently unable to detect any faults in our systems. We recommend that affected Windows customer systems upgrade to version 7.15.18511.0
INC0046
Information on the disruption of Windows systems in connection with Crowdstrike
We are currently unable to detect any faults in our systems. We recommend that affected Windows customer systems upgrade to version 7.15.18511.0
INC0046
June 16, 2024 (Resolved)
[RESOLVED] Storage Migration
Storage migrations were successfully completed. Affected services were affected for max. 1-4 minutes.
INC0045
[RESOLVED] Storage Migration
Storage migrations were successfully completed. Affected services were affected for max. 1-4 minutes.
INC0045
June 16, 2024 (Maintenance)
Storage Migration
We are currently carrying out storage migrations, which may result in short-term disruptions.
INC0044
Storage Migration
We are currently carrying out storage migrations, which may result in short-term disruptions.
INC0044
May 29, 2024 (Resolved)
[RESOLVED] QR Codes are available again (global)
With the release of change M29052024254 and QRCode service, INC0042 is resolved
INC0043
[RESOLVED] QR Codes are available again (global)
With the release of change M29052024254 and QRCode service, INC0042 is resolved
INC0043
May 24, 2024 (Maintenance)
QR codes are not available (global)
All qr codes are right now not available, affecting all services (billing, urlshort and linkme). Solution is in work.
INC0042
QR codes are not available (global)
All qr codes are right now not available, affecting all services (billing, urlshort and linkme). Solution is in work.
INC0042
May 06, 2024 (Resolved)
Delays and interruptions to individual services for European customers
In the morning between 6AM and 9AM, there were isolated disruptions to various cloud services (Mail API, Webmail, URLShorter, CMS) for approx. 35% of our European customers.
INC0040
Delays and interruptions to individual services for European customers
In the morning between 6AM and 9AM, there were isolated disruptions to various cloud services (Mail API, Webmail, URLShorter, CMS) for approx. 35% of our European customers.
INC0040
August 08, 2023 (Maintenance)
Datacenter Network Maintenance
Start: 2023-08-15 18:00 UTC / End: 2023-08-15 22:00 UTC (We do not expect any impact to customer traffic due to this maintenance.)
INC0039
Datacenter Network Maintenance
Start: 2023-08-15 18:00 UTC / End: 2023-08-15 22:00 UTC (We do not expect any impact to customer traffic due to this maintenance.)
INC0039
July 29, 2023 (Resolved)
25Share - Existing shares were temporarily unavailable
Due to an update, existing shares were not available for a period of about 5 minutes.
INC0038
25Share - Existing shares were temporarily unavailable
Due to an update, existing shares were not available for a period of about 5 minutes.
INC0038
June 05, 2023 (Resolved)
Corporate websites no image loading
There were errors loading images and content on our company websites. These are now completely fixed, if necessary, the short-term cash must still be deleted.
INC0037
Corporate websites no image loading
There were errors loading images and content on our company websites. These are now completely fixed, if necessary, the short-term cash must still be deleted.
INC0037
January 23, 2023 (Resolved)
Cloud Management Suite - Login issues
Starting 3AM until 6AM on Jan 23 multiple users had login issues while login to the Cloud Management Suite. The issue is resolved, other systems where not affected.
INC0036
Cloud Management Suite - Login issues
Starting 3AM until 6AM on Jan 23 multiple users had login issues while login to the Cloud Management Suite. The issue is resolved, other systems where not affected.
INC0036
June 21, 2022 (Resolved)
Cloudflare Service Issues
Due to network problems with 3rd party providers, there were isolated accessibility problems in the 9-10AM range.
INC0035
Cloudflare Service Issues
Due to network problems with 3rd party providers, there were isolated accessibility problems in the 9-10AM range.
INC0035
March 25, 2022 (Maintenance)
Maintenance time frame - CLOUD MONITORING
Due to important product enhancements and general maintenance work, there may be short-term monitoring outages for Cloud Monitoring on Saturday (26/03) in the period 9-12 CET. The periods will be short-term monitoring pauses of a few minutes (if at all).
INC0034
Maintenance time frame - CLOUD MONITORING
Due to important product enhancements and general maintenance work, there may be short-term monitoring outages for Cloud Monitoring on Saturday (26/03) in the period 9-12 CET. The periods will be short-term monitoring pauses of a few minutes (if at all).
INC0034
February 02, 2022 (Investigating)
Micro services metric issues
We are currently facing issues with the metrics of different micro services. Also the virtual server monitoring is currently disturbed.
INC0033
Micro services metric issues
We are currently facing issues with the metrics of different micro services. Also the virtual server monitoring is currently disturbed.
INC0033
January 21, 2022 (Investigating)
Mail delivery on technical domains on25space.com
We are currently investigating issues on the technical domain on25space.com - MailAPI is not affected.
INC0032
Mail delivery on technical domains on25space.com
We are currently investigating issues on the technical domain on25space.com - MailAPI is not affected.
INC0032
November 15, 2021 (Resolved)
Degraded performance in some cases for WebApp websites
We have noticed temporary problems with the loading times of isolated WebApps in the period of the last 20 minutes. All services are fully running again.
INC0031
Degraded performance in some cases for WebApp websites
We have noticed temporary problems with the loading times of isolated WebApps in the period of the last 20 minutes. All services are fully running again.
INC0031
August 02, 2021 (Identified)
For multiple mail accounts cookie/cache deletion may be necessary.
When using multiple mail accounts in the webmail client, it may be necessary to clear the browser cache and cookie if login is not possible. Affected users will receive a message that the login is not necessary.
INC0030
For multiple mail accounts cookie/cache deletion may be necessary.
When using multiple mail accounts in the webmail client, it may be necessary to clear the browser cache and cookie if login is not possible. Affected users will receive a message that the login is not necessary.
INC0030
July 13, 2021 (Resolved)
Critical database connection issues
Today we had multiple cricital issues by connecting against our database cluster between 8:26-8:55 CEST. The issue is resolved globally and reported.
INC0029
Critical database connection issues
Today we had multiple cricital issues by connecting against our database cluster between 8:26-8:55 CEST. The issue is resolved globally and reported.
INC0029
July 08, 2021 (Resolved)
Database metrics temporary not available
Starting at 8:20 CEST the database metrics where not available for about 20 minutes
INC0028
Database metrics temporary not available
Starting at 8:20 CEST the database metrics where not available for about 20 minutes
INC0028
July 01, 2021 (Resolved)
25Share upload issue
We have resolved an issue where some uploads are afterwards with 0KB displayed and not usable.
INC0027
25Share upload issue
We have resolved an issue where some uploads are afterwards with 0KB displayed and not usable.
INC0027
May 20, 2021 (Resolved)
Investigating metrics for databases temporary not available
We had solved an issue, that metrics where not available for database-services today from 8:15 to 8:30 CEST.
INC0026
Investigating metrics for databases temporary not available
We had solved an issue, that metrics where not available for database-services today from 8:15 to 8:30 CEST.
INC0026
May 12, 2021 (Resolved)
Cloud Management Suite - Login not possible
The login to the Cloud Management Suite was not able for about 2 minutes from Europe countries.
INC0025
Cloud Management Suite - Login not possible
The login to the Cloud Management Suite was not able for about 2 minutes from Europe countries.
INC0025
April 27, 2021 (Resolved)
[RESOLVED] Loading time-outs for WebApp services
Issue is resolved
INC0024
[RESOLVED] Loading time-outs for WebApp services
Issue is resolved
INC0024
April 27, 2021 (Investigating)
Loading time-outs for WebApp services
Our engineering team is investigating this issue.
INC0023
Loading time-outs for WebApp services
Our engineering team is investigating this issue.
INC0023
April 18, 2021 (Maintenance)
25Share maintenance resolved
The new generation is now global available
INC0022
25Share maintenance resolved
The new generation is now global available
INC0022
April 18, 2021 (Maintenance)
Maintenance – 25Share global services
We are currently performing extensive maintenance work globally at 25Share. The services are currently not available.
INC0021
Maintenance – 25Share global services
We are currently performing extensive maintenance work globally at 25Share. The services are currently not available.
INC0021
February 09, 2021 (Resolved)
Webmail is experiencing degraded performance
At approx. 5:20 pm, we noticed problems for a few minutes when calling up the webmailer. Fixed.
INC0020
Webmail is experiencing degraded performance
At approx. 5:20 pm, we noticed problems for a few minutes when calling up the webmailer. Fixed.
INC0020
January 27, 2021 (Maintenance)
Migration of CDN and DNS network infrastructure
We are currently migrating our DNS domain structure of our services to a new generation. We do not expect any disruptions, only longer loading times in the short term.
INC0019
Migration of CDN and DNS network infrastructure
We are currently migrating our DNS domain structure of our services to a new generation. We do not expect any disruptions, only longer loading times in the short term.
INC0019
December 30, 2020 (Investigating)
25Welcome - Limited range of images
Due to disruptions at a provider, the offer of wallpapers at 25Welcome is currently limited.
INC0018
25Welcome - Limited range of images
Due to disruptions at a provider, the offer of wallpapers at 25Welcome is currently limited.
INC0018
December 14, 2020 (Resolved)
Google Cloud Services outage
Based on the Google Cloud Services outage, there may have been isolated issues with Managed Google Services (ordering and management) as well as push notifications.
INC0017
Google Cloud Services outage
Based on the Google Cloud Services outage, there may have been isolated issues with Managed Google Services (ordering and management) as well as push notifications.
INC0017
November 29, 2020 (Resolved)
Short-term restrictions in different systems.
On the morning of November 29th, we were notified of various faults such as the Cloud Management Suite, Mail and Welcome. The problems have now been completely resolved and all systems are working flawlessly again.
INC0016
Short-term restrictions in different systems.
On the morning of November 29th, we were notified of various faults such as the Cloud Management Suite, Mail and Welcome. The problems have now been completely resolved and all systems are working flawlessly again.
INC0016
November 14, 2020 (Resolved)
Incorrect display of call numbers for URLShort
We have found discrepancies in the number of hits on the URL Short. These are now completely fixed.
INC0015
Incorrect display of call numbers for URLShort
We have found discrepancies in the number of hits on the URL Short. These are now completely fixed.
INC0015
October 26, 2020 (Resolved)
Temporary timeout on different services for about 1 minute
This morning around 5:20am we had short timeouts on different services (CMS, Webmail, Cronjob, URLShorter) of up to 1 minute. The malfunction has been completely fixed, there are no aftereffects.
INC0014
Temporary timeout on different services for about 1 minute
This morning around 5:20am we had short timeouts on different services (CMS, Webmail, Cronjob, URLShorter) of up to 1 minute. The malfunction has been completely fixed, there are no aftereffects.
INC0014
September 28, 2020 (Resolved)
Image loading issue 25Welcome
Our engineering team resolved a temporary issue about loading background images for 25Welcome
INC0013
Image loading issue 25Welcome
Our engineering team resolved a temporary issue about loading background images for 25Welcome
INC0013
September 6, 2020 (Resolved)
Longer latency at 25Me
We have noticed longer loading times for the service 25Me and LinkMe for about 2-3 minutes. The problem is solved.
INC0012
Longer latency at 25Me
We have noticed longer loading times for the service 25Me and LinkMe for about 2-3 minutes. The problem is solved.
INC0012
August 6, 2020 (Resolved)
Endless loop at login - CMS
We have identified a short term problem where users have run into an infinite loop when they log into the Cloud Management Suite.
INC0011
Endless loop at login - CMS
We have identified a short term problem where users have run into an infinite loop when they log into the Cloud Management Suite.
INC0011
August 3, 2020 (Resolved)
Short timeout for URL Shorter
We can confirm a time-out, about 1 minute affected all URL-Shorts (new and existing). The issue is resolved.
INC0010
Short timeout for URL Shorter
We can confirm a time-out, about 1 minute affected all URL-Shorts (new and existing). The issue is resolved.
INC0010
July 10, 2020 (Resolved)
Impacting Managed Databases
Our Engineering team has implemented a fix to resolve the issues about short latency errors when try to access or manage Databases across all regions. We will continue to monitor the situation closely, and will post an update.
INC009
Impacting Managed Databases
Our Engineering team has implemented a fix to resolve the issues about short latency errors when try to access or manage Databases across all regions. We will continue to monitor the situation closely, and will post an update.
INC009
June 7, 2020 (Resolved)
Data timeout on BI
We confirm a short time database-timeout on the BI systems for Analytics systems „ma01“. The issue is resolved. Start: 6:21pm End: 6:24pm
INC008
Data timeout on BI
We confirm a short time database-timeout on the BI systems for Analytics systems „ma01“. The issue is resolved. Start: 6:21pm End: 6:24pm
INC008
May 7, 2020 (Resolved)
Timeouts on different services
On 07.05. in the period of 8:16 pm and 8:24 pm we were able to determine timeouts on different systems. The timeouts ranged from a few seconds to 2 minutes on different systems. All systems are fully available again since 8:25 pm. Affected systems: Cloud Management Suite MailAPI BI Services 25Welcome URLShorter Cronjob
INC007
Timeouts on different services
On 07.05. in the period of 8:16 pm and 8:24 pm we were able to determine timeouts on different systems. The timeouts ranged from a few seconds to 2 minutes on different systems. All systems are fully available again since 8:25 pm. Affected systems: Cloud Management Suite MailAPI BI Services 25Welcome URLShorter Cronjob
INC007
March 5, 2020 (Resolved)
Timeout issues on WebApps
07:00pm The issue is resolved, our team can’t find further issues in the last 20 minutes. 06:40pm Currently all systems are running and are available – investigation 06:35pm Our engineering team is currently investigate multiple timeouts in webapp services. We will give further updates as soon as possible.
INC006
Timeout issues on WebApps
07:00pm The issue is resolved, our team can’t find further issues in the last 20 minutes. 06:40pm Currently all systems are running and are available – investigation 06:35pm Our engineering team is currently investigate multiple timeouts in webapp services. We will give further updates as soon as possible.
INC006
March 5, 2020 (Resolved)
Documents and bills are not visible in CMS
On the 05th. March 2020 we had global issues to view documents and billings (invoices) in the document viewer, in the Cloud Management Suite. The issue appears between 01pm-02pm and is now already solved.
INC005
Documents and bills are not visible in CMS
On the 05th. March 2020 we had global issues to view documents and billings (invoices) in the document viewer, in the Cloud Management Suite. The issue appears between 01pm-02pm and is now already solved.
INC005
November 11, 2019 (Resolved)
Maintenance – Database enviroment
Start: 2019-11-15 20:00:00 pm (UTC) End: 2019-11-16 07:00:00 am (UTC) // We are executing routine maintenance on the service (s) hosting our database services. During the above window, we will be performing migrations of all affected services to ensure no unexpected disruption to service. We will attempt to perform live migrations in all possible cases. A live migration would result in no downtime, but minor performance decreases in disk I/O and a second or less of packet loss as the network is switched over to the new physical host. During the migrations, Cloud Management Suite events will not succeed on the affected service (s).
INC004
Maintenance – Database enviroment
Start: 2019-11-15 20:00:00 pm (UTC) End: 2019-11-16 07:00:00 am (UTC) // We are executing routine maintenance on the service (s) hosting our database services. During the above window, we will be performing migrations of all affected services to ensure no unexpected disruption to service. We will attempt to perform live migrations in all possible cases. A live migration would result in no downtime, but minor performance decreases in disk I/O and a second or less of packet loss as the network is switched over to the new physical host. During the migrations, Cloud Management Suite events will not succeed on the affected service (s).
INC004
November 1, 2019 (Resolved)
Welcome Image error
The services mdctWelcome and 25Welcome had problems with the image display during the last 24 hours. The problem is currently being investigated, new display images have been implemented in the meantime.
INC003
Welcome Image error
The services mdctWelcome and 25Welcome had problems with the image display during the last 24 hours. The problem is currently being investigated, new display images have been implemented in the meantime.
INC003
September 2, 2019 (Resolved)
Short timeout in FRA1 – Management
Our management services in region FRA1 had a short timeout about 1 minute at 21:50 CEST. Affected systems: Cloud Management Suite, Cloud Mail
INC002
Short timeout in FRA1 – Management
Our management services in region FRA1 had a short timeout about 1 minute at 21:50 CEST. Affected systems: Cloud Management Suite, Cloud Mail
INC002
July 31, 2019 (Resolved)
2 Minutes timeout for web interface
Our customer web interface services were partly not available on 31.07. between 9:35pm and 9:37pm. The problem was fixed immediately.
INC001
2 Minutes timeout for web interface
Our customer web interface services were partly not available on 31.07. between 9:35pm and 9:37pm. The problem was fixed immediately.
INC001