devops

yup, EBS allows us to set triggers, so once we start seeing traffic and load (generally on the node machines) it's set to spin up another. Minimizing spin up time has been something we focus on as it increases the responsiveness of the sys

once the load decreases for a time, we spin machines down. And we get alerts whenever these events happen via email, so we can keep an eye on any unusual behavior

Adding to Drew, we don't use sticky sessions, thus it helps to evenly distribute both old and new load as it autoscales.

@CrowdDr Do socket.io sessions move around your servers as well?

This is an amazing application. I'm bringing it to all my clients and watch my competitors in my rear view mirror

adding this to the question: How is Redis deployed in elastic beanstalk for CrowdChat? Is this unique?

Redis is 100% in-memory and is highly performant.

We chose redis over the myriad of options due to its performance and data structures. We use it as a primary data store which is rather unique as most people use it solely for caching.

It offers more advanced data structure support and bigger values to its keys (compared to DynamoDB)

We maintain our own scalable redis system in ec2, while it's not managed with elasticbeanstalk yet there are plans to dockerize it.

What kind of performance are you seeing? Is scale an issue?

our setup is loosely based on how twitter works, with our own improvements of course

the database performance has never been a problem, and we could theoretically scale to the size of twitter with our planned architecture

The socket.io library is actually very good about picking the best transport available for realtime communication based on what is available. Also using https and properly maintaining connection minimizes those sorts of issues

We used https which insulated our protocols from firewall interference. Besides, that socket.io is "known" to switch to best transport.

The key challenge was DNS redirection to ensure everyone is using https transport. It was difficult since we don't have direct access to load balancer directly (these are some pain points in using ELB)

EBS adds flexibility, simplifying automation of deployment across many machines and scalability based on custom triggers

By comparison, services like heroku can get more pricy and more difficult to maintain. Also we already used many Amazon services, such as sqs and s3

Do you guys leverage both Redis and DynamoDB? how do you get real time working so great? That's a BFD (big freaking deal)

Abstraction from gears which we don't have to handle. It is directly integrated with our code on github. Devs can focus on writing code rather than spending time managing and configuring servers, databases, load balancers, firewalls etc.

We maintain our custom scalable Redis in EC2, DynamoDB did not have the data structures and pubsub support needed for our socket.io channels and realtime events

@drew_bratcher Do you worry about getting locked in to AWS, if you are integrating with all of their higher level services? A lot of folks seem to try to insulate themselves using libraries such as libcloud / jcloud.

We primarily use Redis because it is more real-time than DynamoDB and it supports variety of data-structures and bigger data values for keys.

@sirishraghuram it's true a bit of the code would need to be rewritten, but I like to think our code is modular enough that it would simply be a matter of rewriting a few files and ip references

@sirishraghuram In general liblcoud/jcloud seems like extra overhead and code that you would have to maintain. Theoretically you should not be switching clouds often, that'd be couterproductive

@sirishraghuram Thats no real insulation with any library. Elasticbeanstalk does not require us to code anything differently. It just makes our deployment and maintenance easier

@sirishraghuram ELB also integrates with Docker which people now believe is most portable/insulated solution

@CrowdDr My point wasn't about ElasticBeanStalk in particular but a broader question around AWS Service APIs. We internally use tools like boto and when it came time to running our dev workloads against #OpenStack, it took longer than ideal.

@sirishraghuram Oh yea I hate boto, I've tried to use it in the past with much pain. In general node has npm modules which make our life a breeze. Other clouds may not have as much support there as AWS of course

@drew_bratcher Node is awesome. Just npm is heaven for anyone who's had to deal with Python packages. I wish we could do a lot more node!

We use 100% Asynchronous code cc /@drew_bratcher

The biggest challenge is handling the mayhem that results due to such massive parallel execution paths.

Maintaining stability while trying to get multiple oauth network api's to play nicely with each other has also been a challenge...

Why is it so hard to unify the users in real time on one thread? What is the large scale problem you solve? What are some benchmarks?

It comes down to whether you feel the need for a private cloud. AWS is a fantastic platform to start, but in our experience it is expensive. If you are spending $100k per month, you are likely better off with some private cloud.

And Platform9 is probably the easiest and most cost effective private cloud option there has ever been.

@sirishraghuram Zynga is switching back to AWS after leaving to build their own data centers. CIA uses AWS. Integrated stacks are the key to #devops imo

@furrier For every example that is switching back, there are probably 10x who are not going back once they hit scale. I agree about integrated stacks though. The truth is that Private Clouds have been really hard so far. But that is changing now.

@furrier CIA doesn't use AWS exclusively, they also use a massive private cloud that is far bigger than their AWS deployment.

@sirishraghuram Apple uses #AWS for icloud what tech do you have to get developers excited? IBM is trying hard with bluemix now too so is HP

@furrier is bluemix a private cloud? or just an AWS clone?

@furrier Being based on #OpenStack, we're an API first system. So, Infrastructure as Code, just like with AWS. Within 5 minutes, you get Compute and Image service APIs working. From there on, you can provisioning app stacks of your choice via code.

@furrier @drew_bratcher Btw, we use AWS internally as well, and we're fans as well. I am just not crazy about the bills :) We run our production servers on AWS, and all dev/test on Platform9's dogfood environment.

Bluemix is playing catchup and trying to be hybrid AWS and Enterprise ready. AWS is running away with the public cloud. Google is the only thread to it. Microsoft won't get there in my opinion

We use a node.js and redis based stack focusing on realtime events with socket.io. This stack allows our app to be scalable, responsive, and performance oriented.

Like wise the asynchronous nature of stack comes in handy for adding networks and features while retaining the performance at scale

adding to Drew's point. All of this is packed inside ElasticBeanStalk container, which loadbalances and autoscales. For the backedend search we use ElasticSearch. For Queue we use SQS and our proprietary horizontally scalable consumers.

Does AWS help with the threading from multiple networks of users in real time? Did you write code or use AWS tech?

AWS services and abstractions have really allowed the development team to focus on features rather than maintenance and classic IT problems

We use AWS application container extensively. It boxes our server and our software and autoscales. It makes us 100% infra as code.

from a developer's standpoint it does a great job of letting us focus on code rather than system configuration and scale