What is the Biggest IT Cockup You've Ever Made? What is the Biggest IT Cockup You've Ever Made? - Page 2
Page 2 of 5 FirstFirst 1234 ... LastLast
Posts 11 to 20 of 43
  1. #11

    Super poster

    quackhandle is NOT a disguised employee

    quackhandle's Avatar
    Join Date
    Mar 2010
    Location
    Bag End, The Shire, Middle Earth
    Posts
    2,361

    Default

    Realising that SQL Server 2000 object dependency is a bag o tulipe whilst fending off 100s of calls from warehouse managers around the country asking why the parcel management system is still offline after three hours when it was only supposed to be for 30 mins.

    Oh we had a chuckle over that one. (once I'd taken a few beta blockers, natch).

    qh
    He had a negative bluety on a quackhandle and was quadraspazzed on a lifeglug.

  2. #12

    Double Godlike!

    MrMarkyMark is a fount of knowledge


    Join Date
    Dec 2012
    Posts
    10,973

    Default

    I wrote on another post that I once restored <A Large Telecom Providers> Production Reports via my own personal back up on a pen drive.

    There was a little more to the story in that I was helping a user at the time and was viewing a report in Production.
    On exiting the report file I then noticed the whole folder of 200 odd Production reports was missing .

    It seemed the user involved was less than honest and fingers were being pointed my way, the head of IT even said "Are you sure you didn't do it?".

    As I had set up the whole system, unknowingly to the user, full auditing was switched on.
    It was then a simple matter of running a report to show the date and time of the deletion, complete with the users name

    Back up tape machine was broken

    We then restored the whole lot of 200 Prod reports from my own personal pen drive

  3. #13

    Ddraig Goch

    psychocandy - scorchio!


    Join Date
    Apr 2011
    Location
    Cymru
    Posts
    10,145

    Default

    Quote Originally Posted by northernladuk View Post
    I was working late night at a Tambrands site transferring all their drawings in to DXF format using various scripts and batch files. On one of the trips to the server room to set the next load off I though I could improve the throughput by killing some threads off. Worked a treat, the drawings were copying much faster but it was eerily silent. Went back on to the shop floor and some of the automated tampon making machines seemed to have stopped. (gulp).
    Had to call the out of hours guy out to restart it all. Very uncomfortable day next day on site. Got my one and only written warning for that.
    Quite apt you used to work in a tampon factory. ;-)
    Rhyddid i lofnod psychocandy!!!!

  4. #14

    Ddraig Goch

    psychocandy - scorchio!


    Join Date
    Apr 2011
    Location
    Cymru
    Posts
    10,145

    Default

    Can't remember any huge screw ups apart from one. Managed to overwrite part of a DB of an app managing a global network. Customer couldnt manage their network (or reroute it) for hours until it was fixed. Luckily no outages so nothing lost otherwise it would have cost them (they reckon) £1million an hour in costs if they lost links.

    My only ever verbal warning as an employee was a bit of a stitch up. They gave me an expensive sparc laptop (about £10K) when I started. I hardly ever used it so locked it in desk where I was. Moved desk a week later and there was no lockable desk so I left it where I was.

    Week later went to retrieve it - gone. Plus a few of my manuals etc. Of course, an investigation took place and it never turned up. I was a bit miffed to get a verbal warning for "losing it". I even asked HR in the meeting - "So what exactly should I have done in this circumstance?" and they just shrugged their shoulders. I should have appealed but didnt bother in the end.
    Rhyddid i lofnod psychocandy!!!!

  5. #15

    Old Greg is my bitch's bitch

    northernladyuk is a fount of knowledge

    northernladyuk's Avatar
    Join Date
    Nov 2011
    Location
    Working the streets of your imagination
    Posts
    9,460

    Default

    Quote Originally Posted by psychocandy View Post
    Quite apt you used to work in a tampon factory. ;-)
    His lifetime discount on manpons comes in handy.
    Where there's muck there's brass.

  6. #16

    Should post faster

    geoffreywhereveryoumaybe is too good to be a permie

    geoffreywhereveryoumaybe's Avatar
    Join Date
    Apr 2009
    Posts
    124

    Default COBOL Cockup

    Back in the eighties I put my first release live and it all tested fine in the test environment and was to be released the evening before I took annual leave.

    Came back from my holidays to discover that I hadn't release the PSB to accompany my code change and discovered that the live policy system had crashed causing mayhem with nobody having a clue what had gone wrong.

    Learnt one important lesson - never deliver anything important prior to a holiday.

  7. #17

    My post count is Majestic

    vetran is always on top

    vetran's Avatar
    Join Date
    Jul 2005
    Location
    Come Friendly Bombs
    Posts
    31,395

    Default

    I ran rm-rf at the wrong level on a call logger. I said great we can restore from the tape as the system was installed 6 months ago. Oh we wanted you to fix that while you were there the installer never got the tape working.

    Cue a long night building the system again from scratch something that the installer took 3 days doing.

    I then drove back from Dorset about 4am and they wanted me in London at 8.
    "If you didn't do anything that wasn't good for you it would be a very dull life. What are you gonna do? Everything that is pleasant in life is dangerous."

    I want to see the hand of history on his collar.

  8. #18

    I Am Legend

    BrilloPad has reached the peak. Play again?


    Join Date
    Aug 2006
    Posts
    103,786

    Default

    Quote Originally Posted by vetran View Post
    I ran rm-rf at the wrong level on a call logger. I said great we can restore from the tape as the system was installed 6 months ago. Oh we wanted you to fix that while you were there the installer never got the tape working.

    Cue a long night building the system again from scratch something that the installer took 3 days doing.

    I then drove back from Dorset about 4am and they wanted me in London at 8.
    Reminds me about the time at GS when a manager complained about lack of prod disk space. Some youngster suggested \rm -rf * (rm was aliased to ensure you were always prompted). They really got a bawling out for that.....

  9. #19

    Old Greg is my bitch's bitch

    northernladyuk is a fount of knowledge

    northernladyuk's Avatar
    Join Date
    Nov 2011
    Location
    Working the streets of your imagination
    Posts
    9,460

    Default

    I'm not an IT person at all but I did once bring down 'something' on a national scale (Mail or NHS Network - I didn't ask too closely) after creating an auto-reply loop between a GP practice and a hospital which multiplied the number of emails exponentially. It was a really neat piece of clinical service redesign.
    Where there's muck there's brass.

  10. #20

    Prof Cunning @ Oxford Uni

    WTFH is always on top

    WTFH's Avatar
    Join Date
    Nov 2012
    Posts
    17,771

    Default What is the Biggest IT Cockup You've Ever Made?

    Quote Originally Posted by northernladyuk View Post
    I'm not an IT person at all but I've had a cockup or two.
    FTFY
    Strong and Stable Moderation

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •