SKILupDay

   3 months ago
#SKILupDayAIOps and MLOps #SKILupDayDiscussing AIOps and MLOps ahead of SKILup Day on October 15
DevOps Institute
Why should people care about continuous testing?
Marc Hornbeek, DevOps_the_Gray esq.
People should care about continuous testing because without well engineered continuous testing continuous delivery pipelines will have bottlenecks to efficient flow.
felipe dueΓ±as
Continuous testing is a must in continuous integration and continuous delivery
𝓐𝓷𝓾𝓻π“ͺ𝓰 𝓒𝓱π“ͺ𝓻𝓢π“ͺ
Software failures = Headlines, erodes customer confidence and brand So we really need to refine the process in light of business risks as well as software quality. To achieve CT is your friend
OritG
continuous testing is part of the CI/CD pipeline - it ensures that testing is part of any build and makes sure that poor quality code fails the pipeline and does not reach produciton
Mark Peters
Continuous testing enables fast cycles and feedback for your DevOps delivery
Dr. G
The sheer amount of manual toil and cost that continuous testing reduces makes a very compelling argument for continuous testing
Satyan Prakash
Fix the issues at the source where it is generated
Mark Peters
@GO_Orit Always have to make sure we have the right quality set, too many companies start with CT as, did the test run?
Bryan Finster
Continuous delivery IS testing. Testing starts with thought experiments for features and end when we get feedback from production to inform the next test.
TracyRagan
First, they must really consider testing core to DevOps, and if you are doing DevOps @ Scale, you need continuous testing.
Mark Peters
@mhexcalibur And identifying bottlenecks would be a core requirement for good DevOps?
Dr. G
Value is generated only when it successfully makes the customer happy. Continuous testing helps drive that value with lesser friction and more velocity
Satyan Prakash
Shifting the test to the left will make sure the cost of fixing issues is cheaper
Mark Peters
@anuragthoughts How would you define the difference between testing for risk and testing for code quality?
Eric Chapman
Continuous Testing insures you are building in quality instead of testing in quality.
Vishnu Vasudevan
CT is a important component of continuous delivery. Testing complex applications, products, and services can
become a challenge as testing needs to make sure that high-quality software is delivered at high velocity.

(edited)

Maciej Jarosz
If a process is to be efficient then it needs continuity. That's on a small scale. Scale that up and one will need to set up interfaces between different processess for them to be effective. Feedback loops must be set up in place if a decent decision is to be made.
Dr. G
@ericrmchapman +1 to that. Testing becomes part of our culture and not an after thought
Mark Peters
@BryanFinster Like hypothesis driven development? Or... test driven development? Define the problem, decide how you want to verify and write the test.
Bryan Finster
@TinyCyber It depends on your definition of quality code. For me, quality code is secure and easy to change. The first is a tooling issue. The second is measurable by the ability to deliver changes quickly with that code.
Satyan Prakash
break the testing silos, everybody is responsible for testing
Santanoo Bhattacharjee
evolving stages of developmental artefacts are more fine tuned
Bryan Finster
@TinyCyber Yes, HDD. However, even if you aren't predicting value, you can start testing with the BDD conversation. "How should this work?"
Maciej Jarosz
Shif the testing responsibility left - from customers to developers in general.
Satyan Prakash
the speed at which you are doing development in agile iterations continuous automated testing is mandatory
𝓐𝓷𝓾𝓻π“ͺ𝓰 𝓒𝓱π“ͺ𝓻𝓢π“ͺ
@TinyCyber my views anything endanger product/service goal is test for risk and such adjacement CT can address. code quality can be easily address by engineering practice. RBT is also a good option
Mark Peters
@BryanFinster hmm, so standardized review like sonarqube and other SAST are the measurement? Or is it code re-use along the way?
Bryan Finster
@TinyCyber static analysis gives you one perspective, but you can measure the impact of poor quality code by how long simple changes take to complete. Delivery slows.
Santanoo Bhattacharjee
a major challenge is of measuring the distributed applications because of the varied set of dependencies they bring within. how do we address this void, progressively?
Mark Peters
@BryanFinster One of those definition terms, "simple" vs complex,
DevOps Institute
How do we measure whether testing is effective?
Dr. G
Defect Escape Rate
Shivagami Gugan
number of rollbacks?
Dr. G
Defect Escape Rate is a good measure
Marc Hornbeek, DevOps_the_Gray esq.
Effective tests are good at finding failures and are not fragile. Tests that always pass or break are not effective.
Dr. G
Number of "feature requests" that pop in after each release
Eric Chapman
How do we measure whether testing is effective? Deploy it to PROD....frequently!
Bryan Finster
The proof is in the pudding. Tests need to be tested and that occurs when the code is deployed. Effective tests reduce defects. Track delivery frequency plus defect rates and change fail rates.
Mark Peters
Value! Did the testing process increase the value of the delivery
Vishnu Vasudevan
Defect volume, Defect escape rate
Maciej Jarosz
Statistics. Defects pushed into production, bugs discovered at separate stages of software development process. The amount of workarounds that has been pushed to production due to technical debt earlier.
TracyRagan
To measure the effectiveness of anything is to know the KPIs. For example, set a goal to achieve a specific number of incident reports you want to achieve. Measure that one KPI to determine your effectiveness.
Shivagami Gugan
@gpallapa could be due to requirements stability too
Bryan Finster
@TinyCyber Only if the feature is valuable. :)
Dr. G
Variance between the number of bugs closed "working as designed" and the bugs seen in production :)
Satyan Prakash
ratio of defects in various SDLC phases
Dr. G
Totally agree!
Mark Peters
@BryanFinster If the feature isn't valuable, we shouldn't be building it?
Vishnu Vasudevan
@gpallapa Is that because of not having right test data in lower environments ?
𝓐𝓷𝓾𝓻π“ͺ𝓰 𝓒𝓱π“ͺ𝓻𝓢π“ͺ
@DEVOPSINS how much happiness we delivered to user and less pain to Operation. yes Defect escape is one of good measure
felipe dueΓ±as
it is good when you can evaluate business risk coverage.
OritG
If testing is effective you should be able to see a reduced number of production incidents and bugs reported from the users. It is also worth while to triage "buggy" areas to see is flaky code that needs special attention and additional tests to be added
Bryan Finster
@TinyCyber I agree. The statistics show we are bad at predicting this. The more work we do testing before we touch keyboards, the better this gets.
Mark Peters
@BryanFinster Just have to make sure we don't wait for 100% testing solution, need a testing solution to move forward, use feedback, and advance to the next step
Mark Peters
@anuragthoughts How about, how much happiness we bring to the bosses who pay my salary?
Mark Peters
@felidue always have to discuss risk, across multiple venues and business. If we are not discussing risk, we are not making progress.
𝓐𝓷𝓾𝓻π“ͺ𝓰 𝓒𝓱π“ͺ𝓻𝓢π“ͺ
@TinyCyber Bosses can be ignored! lets focus on people who develop, test support and customer who use :D
Mark Peters
@anuragthoughts Those are important and they need to be happy, shouldn't the boss be part of the team?
𝓐𝓷𝓾𝓻π“ͺ𝓰 𝓒𝓱π“ͺ𝓻𝓢π“ͺ
@TinyCyber yes boss is part of team :D he/she will be happy once customer happy. Also love to make bosses at all level (Encourage acts of leadership at every level)