SKILupDay

People should care about continuous testing because without well engineered continuous testing continuous delivery pipelines will have bottlenecks to efficient flow.

Continuous testing is a must in continuous integration and continuous delivery

Software failures = Headlines, erodes customer confidence and brand So we really need to refine the process in light of business risks as well as software quality. To achieve CT is your friend

continuous testing is part of the CI/CD pipeline - it ensures that testing is part of any build and makes sure that poor quality code fails the pipeline and does not reach produciton

Continuous testing enables fast cycles and feedback for your DevOps delivery

The sheer amount of manual toil and cost that continuous testing reduces makes a very compelling argument for continuous testing

Fix the issues at the source where it is generated

@GO_Orit Always have to make sure we have the right quality set, too many companies start with CT as, did the test run?

Continuous delivery IS testing. Testing starts with thought experiments for features and end when we get feedback from production to inform the next test.

First, they must really consider testing core to DevOps, and if you are doing DevOps @ Scale, you need continuous testing.

@mhexcalibur And identifying bottlenecks would be a core requirement for good DevOps?

Value is generated only when it successfully makes the customer happy. Continuous testing helps drive that value with lesser friction and more velocity

Shifting the test to the left will make sure the cost of fixing issues is cheaper

@anuragthoughts How would you define the difference between testing for risk and testing for code quality?

Continuous Testing insures you are building in quality instead of testing in quality.

CT is a important component of continuous delivery. Testing complex applications, products, and services can
become a challenge as testing needs to make sure that high-quality software is delivered at high velocity.

(edited)

If a process is to be efficient then it needs continuity. That's on a small scale. Scale that up and one will need to set up interfaces between different processess for them to be effective. Feedback loops must be set up in place if a decent decision is to be made.

@ericrmchapman +1 to that. Testing becomes part of our culture and not an after thought

@BryanFinster Like hypothesis driven development? Or... test driven development? Define the problem, decide how you want to verify and write the test.

@TinyCyber It depends on your definition of quality code. For me, quality code is secure and easy to change. The first is a tooling issue. The second is measurable by the ability to deliver changes quickly with that code.

break the testing silos, everybody is responsible for testing

evolving stages of developmental artefacts are more fine tuned

@TinyCyber Yes, HDD. However, even if you aren't predicting value, you can start testing with the BDD conversation. "How should this work?"

Shif the testing responsibility left - from customers to developers in general.

the speed at which you are doing development in agile iterations continuous automated testing is mandatory

@TinyCyber my views anything endanger product/service goal is test for risk and such adjacement CT can address. code quality can be easily address by engineering practice. RBT is also a good option

@BryanFinster hmm, so standardized review like sonarqube and other SAST are the measurement? Or is it code re-use along the way?

@TinyCyber static analysis gives you one perspective, but you can measure the impact of poor quality code by how long simple changes take to complete. Delivery slows.

a major challenge is of measuring the distributed applications because of the varied set of dependencies they bring within. how do we address this void, progressively?

@BryanFinster One of those definition terms, "simple" vs complex,

...with a ruler #DevOpsDadJoke

Defect Escape Rate

number of rollbacks?

Defect Escape Rate is a good measure

Effective tests are good at finding failures and are not fragile. Tests that always pass or break are not effective.

Number of "feature requests" that pop in after each release

How do we measure whether testing is effective? Deploy it to PROD....frequently!

The proof is in the pudding. Tests need to be tested and that occurs when the code is deployed. Effective tests reduce defects. Track delivery frequency plus defect rates and change fail rates.

Value! Did the testing process increase the value of the delivery

Defect volume, Defect escape rate

Statistics. Defects pushed into production, bugs discovered at separate stages of software development process. The amount of workarounds that has been pushed to production due to technical debt earlier.

To measure the effectiveness of anything is to know the KPIs. For example, set a goal to achieve a specific number of incident reports you want to achieve. Measure that one KPI to determine your effectiveness.

@gpallapa could be due to requirements stability too

@TinyCyber Only if the feature is valuable. :)

Variance between the number of bugs closed "working as designed" and the bugs seen in production :)

ratio of defects in various SDLC phases

Totally agree!

@BryanFinster If the feature isn't valuable, we shouldn't be building it?

@gpallapa Is that because of not having right test data in lower environments ?

@DEVOPSINS how much happiness we delivered to user and less pain to Operation. yes Defect escape is one of good measure

it is good when you can evaluate business risk coverage.

If testing is effective you should be able to see a reduced number of production incidents and bugs reported from the users. It is also worth while to triage "buggy" areas to see is flaky code that needs special attention and additional tests to be added

@TinyCyber I agree. The statistics show we are bad at predicting this. The more work we do testing before we touch keyboards, the better this gets.

@BryanFinster Just have to make sure we don't wait for 100% testing solution, need a testing solution to move forward, use feedback, and advance to the next step

@anuragthoughts How about, how much happiness we bring to the bosses who pay my salary?

@felidue always have to discuss risk, across multiple venues and business. If we are not discussing risk, we are not making progress.

@TinyCyber Bosses can be ignored! lets focus on people who develop, test support and customer who use :D

@anuragthoughts Those are important and they need to be happy, shouldn't the boss be part of the team?

@TinyCyber yes boss is part of team :D he/she will be happy once customer happy. Also love to make bosses at all level (Encourage acts of leadership at every level)