devops

Clouderati on DevOps #AWS
Special DevOps for #AWS Re:Invent yearly cloud event. We will discuss the magic of CrowdChat on AWS
   10 years ago
#devopsClouderati on DevOpsThought leader conversation on DevOps and Cloud. People and tools - good patterns v. anti-patterns
   9 years ago
#DevopsClouderati on DevOps #AWSSpecial DevOps for #AWS Re:Invent yearly cloud event. We will discuss the magic of CrowdChat on AWS
John Furrier
Q4: Whats Elasticbeanstalk or EBS? Why do you guys like it? What do you do to add value to it for CrowdChat?
Drew Bratcher
EBS adds flexibility, simplifying automation of deployment across many machines and scalability based on custom triggers
Drew Bratcher
By comparison, services like heroku can get more pricy and more difficult to maintain. Also we already used many Amazon services, such as sqs and s3
John Furrier
Do you guys leverage both Redis and DynamoDB? how do you get real time working so great? That's a BFD (big freaking deal)
Crowd Doc
Abstraction from gears which we don't have to handle. It is directly integrated with our code on github. Devs can focus on writing code rather than spending time managing and configuring servers, databases, load balancers, firewalls etc.
Drew Bratcher
We maintain our custom scalable Redis in EC2, DynamoDB did not have the data structures and pubsub support needed for our socket.io channels and realtime events
Sirish Raghuram
@drew_bratcher Do you worry about getting locked in to AWS, if you are integrating with all of their higher level services? A lot of folks seem to try to insulate themselves using libraries such as libcloud / jcloud.
Crowd Doc
We primarily use Redis because it is more real-time than DynamoDB and it supports variety of data-structures and bigger data values for keys.
Drew Bratcher
@sirishraghuram it's true a bit of the code would need to be rewritten, but I like to think our code is modular enough that it would simply be a matter of rewriting a few files and ip references
Drew Bratcher
@sirishraghuram In general liblcoud/jcloud seems like extra overhead and code that you would have to maintain. Theoretically you should not be switching clouds often, that'd be couterproductive
Crowd Doc
@sirishraghuram Thats no real insulation with any library. Elasticbeanstalk does not require us to code anything differently. It just makes our deployment and maintenance easier
Crowd Doc
@sirishraghuram ELB also integrates with Docker which people now believe is most portable/insulated solution
Sirish Raghuram
@CrowdDr My point wasn't about ElasticBeanStalk in particular but a broader question around AWS Service APIs. We internally use tools like boto and when it came time to running our dev workloads against #OpenStack, it took longer than ideal.
Drew Bratcher
@sirishraghuram Oh yea I hate boto, I've tried to use it in the past with much pain. In general node has npm modules which make our life a breeze. Other clouds may not have as much support there as AWS of course
Sirish Raghuram
@drew_bratcher Node is awesome. Just npm is heaven for anyone who's had to deal with Python packages. I wish we could do a lot more node!
John Furrier
Q3: How are the Linkedin/Twitter/Facebook users able to update from any network so real-time? What's the big technology challenge?
Crowd Doc
We use 100% Asynchronous code cc /@drew_bratcher
Crowd Doc
The biggest challenge is handling the mayhem that results due to such massive parallel execution paths.
Drew Bratcher
Maintaining stability while trying to get multiple oauth network api's to play nicely with each other has also been a challenge...
John Furrier
Why is it so hard to unify the users in real time on one thread? What is the large scale problem you solve? What are some benchmarks?
Drew Bratcher
@sirishraghuram Is there any reason we should consider switching to platform9 from AWS?
Sirish Raghuram
It comes down to whether you feel the need for a private cloud. AWS is a fantastic platform to start, but in our experience it is expensive. If you are spending $100k per month, you are likely better off with some private cloud.
Sirish Raghuram
And Platform9 is probably the easiest and most cost effective private cloud option there has ever been.
John Furrier
@sirishraghuram Zynga is switching back to AWS after leaving to build their own data centers. CIA uses AWS. Integrated stacks are the key to #devops imo
Sirish Raghuram
@furrier For every example that is switching back, there are probably 10x who are not going back once they hit scale. I agree about integrated stacks though. The truth is that Private Clouds have been really hard so far. But that is changing now.
Sirish Raghuram
@furrier CIA doesn't use AWS exclusively, they also use a massive private cloud that is far bigger than their AWS deployment.
John Furrier
@sirishraghuram Apple uses #AWS for icloud what tech do you have to get developers excited? IBM is trying hard with bluemix now too so is HP
Drew Bratcher
@furrier is bluemix a private cloud? or just an AWS clone?
Sirish Raghuram
@furrier Being based on #OpenStack, we're an API first system. So, Infrastructure as Code, just like with AWS. Within 5 minutes, you get Compute and Image service APIs working. From there on, you can provisioning app stacks of your choice via code.
Sirish Raghuram
@furrier @drew_bratcher Btw, we use AWS internally as well, and we're fans as well. I am just not crazy about the bills :) We run our production servers on AWS, and all dev/test on Platform9's dogfood environment.
John Furrier
Bluemix is playing catchup and trying to be hybrid AWS and Enterprise ready. AWS is running away with the public cloud. Google is the only thread to it. Microsoft won't get there in my opinion
John Furrier
Q2: What stack does CrowdChat uses? and Why?
Drew Bratcher
We use a node.js and redis based stack focusing on realtime events with socket.io. This stack allows our app to be scalable, responsive, and performance oriented.
Drew Bratcher
Like wise the asynchronous nature of stack comes in handy for adding networks and features while retaining the performance at scale
Crowd Doc
adding to Drew's point. All of this is packed inside ElasticBeanStalk container, which loadbalances and autoscales. For the backedend search we use ElasticSearch. For Queue we use SQS and our proprietary horizontally scalable consumers.
John Furrier
Does AWS help with the threading from multiple networks of users in real time? Did you write code or use AWS tech?
John Furrier
Q1: What makes AWS so unique for the CrowdChat app? What's the high level advantage of using AWS
Drew Bratcher
AWS services and abstractions have really allowed the development team to focus on features rather than maintenance and classic IT problems
Crowd Doc
We use AWS application container extensively. It boxes our server and our software and autoscales. It makes us 100% infra as code.
Drew Bratcher
from a developer's standpoint it does a great job of letting us focus on code rather than system configuration and scale
John Furrier
Hello everyone welcome to the #cloud #crowdchat technology technical talk chat here on CrowdChat
Crowd Doc
Its is a CrowdChat AMA re: CrowdChat recursively :)
Crowd Captain
welcome aboard everyone
Sirish Raghuram
John, Thank you for hosting!