Arthur wrote:

jason wrote:

FreeBSD is a fine choice, and one that many make. That said, for what we call "shared hosting", we are moving to solaris and suse linux.

I'd love to hear more about this.

me too.

are you going to use zfs?

jblaine wrote:

Jason, I'm not quite sure how you can say FreeBSD is a fine choice.

Seems like to me he was saying FreeBSD is a perfectly good operating system, but that for shared hosting we're moving to another platform (because, for us, FreeBSD just hasn't been cutting it.)

jblaine wrote:

But I see this will go nowhere and nobody can provide me a reasonable explanation for how it is a web hosting infrastructure was created around boxes without even fsck-less filesystems.

Please don't turn this into one of those holy wars about how much better journaling is than SoftUpdates, or visa versa. I've worked enough with journaled filesystems (especially VxFS and OnlineJFS) to know the benefits, but also to know the cons. The choice was made, and it served us well for a very long time. It obviously hasn't been serving us well for the last couple of months, so we're going to change to something that works better for us.

And yes, for Solaris we will be using ZFS.

Finally, regarding the little SAN argument that was going on back there, please don't assume that we're doing something a certain way until it's stated by someone on the TextDrive staff. The model that we're moving to, while we're putting the filesystems on a SAN, will be more of a NAS model (using multiple Solaris heads in an HA config) to avoid the need to fsck a large home system in case a front-end server crashes.

Hope that helps. And thank you. :)

By the way, I don't recommend that version of the SuSE book if you want a SuSE book -- I wrote nearly half the book (though I'm the fourth-named author), but I did write it in 1999. There is a much more current version out there. I haven't used SuSE in several years, but I may move all my domains off my existing Debian box and rebuild it to be a SuSE box.

jblaine wrote:

I'm not sure where you (any of you) are getting that I am on an OS religion kick or something.

I never implied (or said) that this was an OS religious war. The core of this discussion was about filesystems, and I was responding to your implication that journaled filesystems are the-one-true-way ("something so fundamental to the notion of server availability"). There are other alternatives to journaling and, while journaling is great and all, it's not always the right solution for a server (and when it's a poor fit, it can make for a very, very poor choice of filesystems.)

jblaine wrote:

At least someone acknowledges the current setup is sorely lacking in this area (the first time anyone has to me).

Of course it does. No one should expect FreeBSD to have a journaled filesystem, so there's no reason to not acknowledge that it lacks one.

jblaine wrote:

The notion, however, that this problem was unavoidable....

Hate to break it to you, dude, but sometimes servers just crash. And sometimes they need to go through a really long fsck. Can't help it. Bound to happen sometimes.

Is that what you've been wanting to hear? Because it sure sounds like you think we've been going around saying things like that, and for the life of me I can't find a single instance where any of us have. Just where in the hell did you get the idea that anyone thought this was inevitable? Please, please, if you can't be bothered to look elsewhere in the forum where we've talked quite openly about our unforseen growth, the resultant problems and how we're going to address them, then at the very least don't go around thinking that we're just following merrily and blindly along after our servers and telling everyone, "Hey! Sorry! It was bound to happen, though! Really! Couldn't avoid it!"

jblaine wrote:

I do it every frigging day. I can't recall the last time, in years, any of our server downtime was caused by anything other than staff.

As a customer who's pretty happy with TextDrive, I'm honestly a bit curious about your statements here.

When was the last time, in years, that you ran as many services on a server as TextDrive does, and granted the same level of access to user accounts that TextDrive does?

Shared hosting is a very different sort of task from most sysadmin work, and introduces a whole bunch of lovely new ways in which things can go mushroom-shaped. You'll notice that the staff have been busily working behind the scenes to establish compromises like the new resource limitations, which allow TextDrive to continue offering the tools we've come to know and love while eliminating much of the ability of users to screw things up.

One or two of our engineers' workstations have crashed over the last year due to failed memory, and they sure as hell weren't staring at fsck for even 5 minutes before it came back up squeaky clean.

And I'm sure that those workstations each had hard-disk arrays of the same magnitude as one of TXD's shared servers, right?

So, perhaps textdrive.com doesn't have the resources to do it, be it staff skill or money

So why are you here? If you really think that, and if you're really as fed up as you claim, I have four words for you:

Dreamhost is that way -->

(and yes, that's a referral link; why not do something nice and make me some money?)

jblaine wrote:

I'll split when I'm ready, ESPECIALLY after the hand-waving bullshit I've gotten in this thread in place of an actual response about what has been going on over the last few months and what is being done to fix it.

Dude, you should consider yourself lucky, last time I vented about beloved Gilford going down all the effing time, the thread was deleted :p So now I just shut up and wait for the über-super-duper SAN cross-over giga-cluster to come, or whatever the hell the TxD folks plan to use to fix this mess, because I know they will.

yea i am kind of missing the whole idea here too. i mean, sure, it's a pain when the server is down when you want to be working on something. totally get that. but why get so bent out of shape about it? it's not the end of the world or anything. "oh my god, davie is down! someone...! quick, save me before it's too late!" i'm sure jason or deirdre would be like "here kyle, have a prostitute" and then i'd have to run away screaming because it was a gay prostitute dressed in a clown outfit and i'm straight and that would put the fear of clowns in me. but regardless, by not hanging out in the forums you miss out on the best part of paying your monthly fee to host at textdrive, the community. that's your loss there, and it sounds like you've made enough enemies already to make your stay unpleasant.

i'd say you maybe need to take a chill pill, sit back, stick it out and wait for things to get better. if you can't wait, then change hosts, if you're doing rails there's really only a few choices. if you don't want to be treated like a developer, and instead just be a customer, then you can always goto one of the commodity hosting places out there. of which none of them give you nearly as much as textdrive gives you.

frustrated or not, i still think that you approached this whole thing very wrong on the forum. you started off good with the email, but you definitely should've either emailed them back, or at the very least been polite on the forum. being like you're being now isn't going to make you friends nor will it make your opinion very respected around here. a large majority of the customers here are very happy, i'm sorry you aren't one of them, and the choice is yours, stay or leave. they've given you some information on how they are going to fix the problems, but either that isn't good enough or you just like arguing.

i'm outta this conversion, i've said my piece, and most people know where i stand around here.

jblaine wrote:

No you're not curious.

What you are, though, is trying to divert attention from the problems at hand that I am pointing out, by questioning MY capabilities and server environment. I'm not going to explain my environment to you in order to refute something as ridiculous as an IMAP process taking a host down. You go right on ahead believing that's par for the course if you'd like.

Actually, I am curious. Gets me into no end of trouble sometimes.

I ran a server for a few years that provided web hosting and email for a few users, so I have a very, very basic idea of what it's like. I know that what goes on here at TXD is several orders of magnitude beyond anything I have experience with.

But... in those few years of administration I ran into problems with software that was buggy, but not in immediately obvious ways. I ran into problems with my users doing things I hadn't anticipated or planned for. I had a couple issues that took damn near forever to track down (several of which turned out to be Sendmail's fault, and when I figured that out I dumped it). When you offer even a very few services, interacting with even a very few users, even the best-laid plans can go awry. When you offer lots of services interacting with lots of users, well, I'd expect the number of possible problems increases exponentially. My experience talking with various people of far greater technical ability than myself bears this out.

So... I'm not saying that crashes and downtime are good, or acceptable, or to be shrugged off. But I am saying that they are not, as you seem to be implying, attributable to administrative error in all cases.

Wait, did you just state that there have been NO RESOURCE LIMITATIONS IN PLACE until recently and this is something I am supposed to be excited about? Please stop. You're making things worse.

When TXD first started up, its clientele mostly consisted of either A) people who knew and liked Dean, and didn't need to do much beyond running $BLOG_APP, or B) technically savvy people who could be trusted to test things before deploying them and understand how to be good neighbors on shared servers. As a result, there was little need for any sort of enforcement of limits on resource use, and so the model worked well for a good while.

But TextDrive has scaled up quite a lot in quite a short time, and that model no longer works. As a result, we get things like resource limits, moves from shared servers running all their own services to clusters which each provide a particular function, etc.

To me this seems reasonable; so long as there was no pressing need for limits, TextDrive did its best not to impose any. So long as there was no pressing need for massive, complicated infrastructure (remember: more complexity, more potential problems), TextDrive stuck with a simple shared-server model. Now there is a pressing need for both, and TextDrive is responding to that need in an appropriate fashion. I fail to see how this is unreasonable.

What part of "fsck is unnecessary" don't you understand? It doesn't matter how big the disk is. Please stop.

I've always enjoyed a good fsck. Especially a good long fsck, I don't get that nearly as often as I'd like. But maybe that's just me.

And depending on the OS and filesystem you're using, fsck certainly can be necessary; journalling is not the be-all and end-all of filesystem technology. That said, the model TextDrive originally had, which worked well for quite a good while, does not work so well at the current scale. Thus, changes. Again, I fail to see how this is unreasonable.

That's okay. You guys win. Nothing is wrong.

That's a lovely straw man you've got there. Now, if he only had a brain...

Because it's a pain in the ass to move? I'll split when I'm ready, ESPECIALLY after the hand-waving bullshit I've gotten in this thread in place of an actual response about what has been going on over the last few months and what is being done to fix it.

What's being done to fix it? Well don't look now, but it appears that some infrastructure changes are on the way. And others have been implemented steadily over the past couple months. I've seen problems come and go, but they've always been solved in a timely and responsible manner, so I trust that they will continue to be in the future.

I've said my peace, yet someone, like you did, keeps dragging this thread along though with more absurd defense of how things are.

You? Peace? Not by a long shot, but so long as you need to vent your spleen I'll be here.

daniel wrote:

I think what is most confusing to me is why, if you weren't happy with the initial response you received, you left your anger and frustration to stew for 5+ weeks rather than contact us back looking for further answers or clarification. Instead you've taken it to the forum, into an unrelated thread, and things have really turned sour. Setting aside the fact that I work here, I just don't know exactly what you are trying to achieve here, or more importantly, what you thought this would get you that a second email wouldn't.

Daniel, the sourness this has turned into is not 1 sided. My side of it is based on the responses I started getting at the end of page 1 - hand waving "protective daddy" defensiveness instead of informational fact and consideration for the fact that I am, at the bottom of it all, not being unreasonable and there IS a problem.

All it took was:

a) This is why bidwell keeps going down. Insert a few sentences enumerating specific items.

b) This is what we're doing near term to address it. Insert a few sentences.

c) This is what we're doing long term to address it. Insert a few sentences with targeted solutions to the exact enumerated problems listed in a).

d) An apology for the problems.

I'm not sure how it is TXD doesn't feel something like that is necessary by now with the ongoing problems. Believe it or not, that's all it would have taken for me to be perfectly content for several more weeks. If the near-term fixes were not working, I would expect another notice of some new ideas to get the setup stabilized.

Instead I get into a ridiculous exchange that COMPLETELY falls on deaf ears with staff who are "hurt" they are getting called out.

I should not have to EMAIL, let alone EMAIL then post in a forum weeks later, in order to find this information out as a customer on the receiving end of bidwell's instability. But hey, what do I know. That's just me and how I do things with my own users at work. Apparently my expectations are too high by asking for an 8 on a scale of 10.

I should not, also, have to dick around in the forums every day to keep tabs on the TXD State of the Union.

I'm the guy who bitched a year ago about the Status mailing list never being used. Voila! Someone saw the light and set its usage into policy apparently! RSS feed! And it was a change for the better! And I know at a minimum emergency information now!

I'm sorry I'm not blissfully ignorant and in awe of the TXD staff to sit idly by and "chill out". I'm always blown away by customers/consumers who think that people rightfully displeased with the service or product in question... are "uptight" and should "chill out". It's no wonder customer service in America sucks ass these days. Nobody bothers to speak up. But I digress.

I know now what to expect from TXD.

jblaine wrote:

I should not, also, have to dick around in the forums every day to keep tabs on the TXD State of the Union.

I'm the guy who bitched a year ago about the Status mailing list never being used. Voila! Someone saw the light and set its usage into policy apparently! RSS feed! And it was a change for the better! And I know at a minimum emergency information now!

I'm sorry I'm not blissfully ignorant and in awe of the TXD staff to sit idly by and "chill out". I'm always blown away by customers/consumers who think that people rightfully displeased with the service or product in question... are "uptight" and should "chill out". It's no wonder customer service in America sucks ass these days. Nobody bothers to speak up. But I digress.

I know now what to expect from TXD.

There's really just one rule at textdrive, and that's at every level, and that is Don't be an asshole. Some rephrase it as Don't be a dick. It's not really about awe, or being uptight, or being chill, it's simply about being a dick.

I don't think it's necessary at any level. It's really no different then yelling "fat fuck" at a fat person on a treadmill.

jblaine wrote:

We're here on page 2, and with this tone now, because of your 1 Rule Policy.

Perhaps you need some more.

can someone please close this thread. it's off topic for this thread, it's just a giant bitchfest on one person's part, and what needs to be said has been said. all that's going to happen is more bitching and more complaining and more sour attitudes.

EDIT: if more needs to be said, i'm sure he can take it to email where it's not involving others who can simply give him more to complain about.

Last edited by UnLogikal (Today 14:45:34)

Rules? Accountability? Communication? I'll say it again:

All it took was:

a) This is why bidwell keeps going down. Insert a few sentences enumerating specific items.

b) This is what we're doing near term to address it. Insert a few sentences.

c) This is what we're doing long term to address it. Insert a few sentences with targeted solutions to the exact enumerated problems listed in a).

d) An apology for the problems.

I'm not sure how it is TXD doesn't feel something like that is necessary by now with the ongoing problems. Believe it or not, that's all it would have taken for me to be perfectly content for several more weeks. If the near-term fixes were not working, I would expect another notice of some new ideas to get the setup stabilized.

But here we still are. And I'm just a dick. A dick who started this off the right way, via email, and a dick who still has yet to hear anything worthwhile about the problems and their resolutions, but has heard a lot of defensive yap and insinuation that what I'm suggesting is "impossible".

THAT'S your reply?

Jason Hoffman, ladies and gentleman. President and COO of textdrive.com. One for the books.

Yes, I've clearly breached the Acceptable Use Policy by posing questions and criticism.

So you're reduced to nitpicking a word now, ignoring everything else I have mentioned about forums, communicating with users, etc.

Boy, you really got me on that one. Really clever. I will now go install a screen reader. Ha ha. Oh, my side.

slow golf clap

It's clear this is, sadly, going nowhere. Apparently I've hurt peoples' feelings by being critical. Funny, the only person who hinted at remotely heading in the right direction for this thread's progression was Dierdre by indicating vaguely for half a second you've been having trouble with FreeBSD 5.4's stability. Turns out she doesn't have testosterone blocking her brain.

For those watching from home, this is your hosting company.

I have received email from jan@textdrive.com that my account will be cancelled this weekend, Thanksgiving, without any time to move my data and find new hosting.

Let it also go on record, that I have not in any way whatsoever violated any agreement with Textdrive.com as a paying customer, who paid in full for a year, and has never been a problem.

I voiced my displeasure and for that my account is being cancelled, with notice 4:20PM the day before Thanksgiving, "this weekend".

UnLogikal wrote:

the thing to notice is that you started this argument. yes, you can crack jokes about me saying.. "But mommy, he started it!" .. but whatever, the point is you came in here with your head up your ass trying to whine and complain your way into a "we're really sorry mr. jblaine, we're having some problems and here is our entire set of plans and all of the reasons why things aren't working." ... the thing is, MOST people are happy here, there's a couple who aren't, you included. no one should have to bow down to an arrogant pompous ass who demands things and comes in with a really pissy attitude. you approached it all wrong, had you politely asked and not bitched you might have had better results. how's that for "lack of communication with users?" maybe it's YOUR communication that needs help, the staff are very willing to give you information if you ask for it. you however demanded it and drove it down their throats. not a good way to start a conversation man.

My communication well before this point we've gotten to is fine.

I have NEVER asked anyone to bow down. I asked, as clearly stated above, for information. And, as I have already mentioned, had it been done the first time (when I emailed politely), I would never have even started posting here. Instead, I got bidwell crashing more. As a customer, yes, I was clearly miffed in my first post. Is the entire rest of this thread the way to handle a customer who is rightfully pissed off at one of your hosts bouncing up and down, taking nearly an hour to come up every time?

This thread should, starting at my post, should have been 1 or 2 posts more. And nobody has to "bow down".

I addressed it. I emailed out of courtesy first. I got jack. A day or two later, I submitted tickets about IMAP being down on bidwell. A week or so later, bidwell is tits up and fscking its disk again. I'm really sorry you consider it unacceptable for me as a customer to be ticked off at that and post finally asking for SOMEONE, ANYONE, to give me an explanation and consolation that the obviously broken setup is being investigated and solutions are being developed near-term and long.

I'm sure if your broadband connection stopped working for hours at a time, repeatedly over months, and politeness got you nowhere, that you would be just a happy calm butterfly.

Especially when yelling to get SOMEBODY to wake up and have a conscience about it results in "We're cancelling your account."

Totally unreal.