Problems Scaling App Engine – Web Development

Problems Scaling App Engine – Web Development

It’s not all perfect and easier and improved. Maybe we could talk about some of kind of the downsides, the things that are a little more difficult in App Engine as a result of how it’s set up and how it’s designed. The happy path is paved with concrete, and it has chains along the side, and so you really can’t leave the happy path. So that’s something you have to keep in mind. And there are failures from time to time. A query can work just fine, and then all of a sudden the next time you make exactly the same call, it doesn’t work. And those are just things that you get in a big, eventually consistent kind of amorphous sort of machine. And so you have to deal with those, and so you start to a lot of times kind of want to do a looping sort of query where you’re willing to do the query 3 times. And within the 3 times, you’ll probably get the result, and the first one might fail. You just re-perform the same query before you let your request handler send an error response back to the user, which is actually kind of a good pattern because distributed computing, you’re going to encounter failures. And you need to program defensively for those, like, system sort of failures, so being ready to do an issue and a retry and putting that into your normal call stack or your framework, I guess. It’s easy to do, but it is something that when you really want your app to start running smoothly. And those are the sorts of things Udacity is dealing with or learning about now– to do the retries on things like queries. You have to be cognizant of how long a long-running task on a back end is going to take. You get a limit of–I think it’s 10 minutes, which seems like a long time, but on a highly virtualized system, you can actually have a 10-minute long-running task. So you have to be prepared to shut it off early and then be ready to pick it back up with a subsequent request. One of the other things, I guess, would be a lot of people, when you get a bigger Python system, you want to have C modules pre-compiled, and you don’t get that. So you kind of forget about those sorts of things. And then the biggest one, which I know that Google is addressing– and they have a beta program, and I’m very hopeful that it happens soon– is better SSL certificate support, especially for the custom domain. You get SSL on, but if you point a custom domain to it, you don’t get SSL support for that, which is a big drawback, and it’s something they obviously recognize and they are fixing. So hopefully in time–please, Google–that will be there. So one thing that I was skeptical about coming in is the database and how much access you have to your own data. Are there any limitations there that need to be thought about? Definitely. It’s something that’s probably worthy of a whole unit in a course about going deeper with App Engine. There are a few things that you kind of have to come to terms with, and they’re trade-offs. Every life is trade-offs, especially with scaling a website. And for a lot. You have to put up with things like all queries have a limit of 1000 records. And that’s before paging, which kind of sucks. So you could potentially pull out a result set that was 1000, none of–with just empty records. And then you have to page to the next one to get the 3 that match depending on how it is that the indexing and your collections or your model is structured. The backup and restore thing is real awkward. There’s some backup and restore utilities built in but not if you wanted to just take the data on a nightly basis and sync it to a testing kind of QA, sort of, like a Prod-QA sort of version of your app. There are some tools out there that basically allow you to export the data as Python code, and then you could run it and import it into another app. But in the back of your head– On the one hand, it’s nice to have some transparent costs. We can tell you pretty much exactly how much every one of our queries costs, which is great. The downside is that when you’re doing something like a backup, in the back of your mind you’re going, “Man, this is costing a lot of money.” And so it’s things you can get around but not as smoothly as just taking a hot backup of your local files on a MySQL database or something like that. The other one is that there’s no real great data viewer. There are times when you’re troubleshooting you really don’t know what’s going on. You want to just issue some SQL queries or some Mongo queries directly against the data to see what the data looks like. There is a data viewer on the dashboard, but it’s not that great, and it only works with certain kinds of data types, and you can’t really update anything. So if you really just need to flip some field–the status of something– and that way you can keep processing and then you can work on fixing your bug and getting it out later that day or tomorrow or whatever, you can’t do that, which is frustrating. But on the other hand, it might force you to build some more administrative kind of features that if you can fix things easily with some duct tape, you end up kind of not really building a good administrative interface. So sometimes the only way you can do it is by building an administrative interface. Okay, yeah. So some of those limitations seem to not be without reason, but they are limitations.>>Yeah. It was funny when we were talking before this. Some of the Udacity guys had mentioned that if they were building a website that needed to scale to many, many users, App Engine would be a great choice. And then I kind of mentioned if I was building a toy website, App Engine would be my first go-to so I don’t have to deal with any of that crud of installing databases and memcached and app service and frameworks and all that stuff. Setting up a deploy process, all that.>>Oh yeah. Deployment–man, that’s something we haven’t touched on in this or in any of my units is when you start having multiple machines, deploying code across multiple machines is something that you have to think about. And there’s different strategies, and the versioning stuff is cool in App Engine. So I want to thank you for doing this with us. This is really cool. I’ve learned a lot from talking to you guys about how App Engine works, and I hope the students have got some better perspective on whether App Engine is the right choice for them. Absolutely. Thank you for having me, and thank you for your class. We’ve had really great feedback, and it’s been fun having this sort of class.>>Cool. Glad to hear it. And so that wraps it up for Unit 7, which wraps it up for the whole course. So thank you, everybody, for participating this far. If you made it this far, you are a champion of web applications. And the only thing left for you is the final, which we’ll get to next.

Be the first to comment

Leave a Reply

Your email address will not be published.