Testing and Evaluation: Our Protocol for Espresso Machine Testing

Why a standardized test protocol?

When buying an espresso machine, you face an overwhelming selection: From the compact thermoblock machine for 400 euros to the dual boiler monster for 4,000 euros – how do you compare? That's exactly where our test protocol comes in.

We don't just want to tell you "This machine is great" or "That one is bad". Our goal is to provide an objective, measurable, and comparable evaluation based on hard facts. The result in the cup takes center stage, complemented by measurable technical parameters and a fair economic assessment.

All results are rated on a scale of 0 (unsatisfactory) to 10 (outstanding) and then weighted into an overall score.

Overview of all espresso machines tested by us.

How you should read this protocol

1. Look at the overall score first – it gives you a quick overview of overall quality
2. Identify your personal priorities – What matters to you? Espresso potential? Steaming performance? Energy efficiency?
3. Focus on the relevant individual scores – Ignore categories that are unimportant to you
4. Compare only machines in similar price ranges – A score of 7.5 at 800 euros can be better than 8.0 at 2,500 euros

The weighting: Not all criteria are equally important

To do justice to the different relevance of the aspects, the categories flow into the overall rating with different factors:

Factor 3 (High – The Core)

Espresso potential (3×): The most important criterion. How good is the espresso that can be maximally extracted from the machine? Here technical prerequisites merge with sensory validation.

Factor 2 (Medium – The Technology & Haptics)

These categories significantly influence daily experience and technical performance:

• Temperature management (2×): Consistency and adjustability
• Volumetrics (2×): Precision of automatic portion programming
• Steaming quality (2×): Performance and speed when steaming
• Build quality (2×): Materials, workmanship, and durability
• Usability (2×): Workflow, cleaning, and ergonomics
• Heat-up time (2×): Time to full operational readiness
• Power consumption (2×): Energy efficiency during operation

Factor 1 (Low – Supplements & Special Categories)

These aspects are relevant but less decisive for most users:

• Noise (1×): Sound output during operation
• Accessories (1×): Quality and scope of included items
• Volume potential/Catering (1×): Performance capability with multiple consecutive shots
• Price-to-performance ratio (1×): The economic classification

Overall score calculation

The sum of weighting factors is 21: one factor 3, seven times factor 2, four times factor 1. The result is a grade between 0 and 10.

The quality classes

Depending on the overall score, we assign the machine to a quality class:

The heart of our test

Why this category is the most important

In the end, only one thing matters: How does the espresso taste? A machine can be technically impressive, but if the result in the cup disappoints, everything else is secondary. That's why espresso potential counts with Factor 3 in the overall rating – more than any other category.

Our hybrid approach: Technology + Sensory

Espresso potential results from two pillars:

• 1. Technical prerequisites (max. 8 points) – objectively measurable
• 2. Sensory validation (max. 2 points) – panel test

The technical prerequisites (8 points)

A. Temperature Performance (0-3 points)

The score from Chapter 6 (Temperature Management) is used as the basis here:

• Score ≥ 9.0 in Ch. 6 → 3.0 points
• Score 8.0-8.9 → 2.5 points
• Score 7.0-7.9 → 2.0 points
• Score 6.0-6.9 → 1.5 points
• Score 5.0-5.9 → 1.0 points
• Score < 5.0 → 0.5 points

Temperature stability is the foundation for good espresso. A machine with excellent temperature management already has a major advantage.

B. Profiling & Brew Control (0-3.5 points)

Manual control (0-3 points):

• 3.0 pts: Flow + pressure control: everything possible. Formulation of if-conditions, stages, volumetric control
• 2.5 pts: Programmable pressure profile control
• 2.0 pts: Pre-infusion adjustable (time/pressure) OR simple pressure profiles
• 1.5 pts: Fixed (sensible) pre-infusion present
• 1.0 pts: Manual profiling via paddle/lever possible
• 0 pts: No profiling options

Smart Profiling Bonus (+0.5 points):

Some modern machines feature adaptive systems that monitor flow during extraction and intervene automatically. If the machine detects, for example, too fast a flow (indicator of too coarse a grind), it reduces pressure or extends pre-infusion automatically. This greatly increases fault tolerance and enables even less experienced users to consistently achieve high-quality results.

Maximum: 3.5 points

C. Hardware Hygiene (0-2 points)

Shower quality & water distribution (0-1 point):

• 1.0 pts: High-quality shower, high-quality screens, even distribution
• 0.5 pts: Solid standard shower, screens of medium quality
• 0 pts: Poor distribution, prone to channeling

Thermal mass & pressure build-up (0-1 point):

• 1.0 pts: Fully heated metal basket, stable pressure build-up
• 0.5 pts: Partial thermal mass or slight pressure fluctuations
• 0 pts: Cold basket or unstable pressure

The sensory validation (2 points)

The technical values are the foundation – but do you taste it?

Panel test by Q-Grader Benjamin Hohlmann with reference coffees whose ideal profile is known.

Evaluation:

• +2 points: Machine fully realizes technical potential. Espresso shows balanced harmony, no extraction faults
• +1 point: Technical potential is largely realized, minimal deviations
• 0 points: Clear discrepancy between technology and cup result

We evaluate the machine's ability to produce fine-pored, homogeneous microfoam for latte art. This assessment is done by our barista professional Michel Indelicato, who has steamed with around 100 different machine types in test scenarios.

The categories (max. 10 points)

A. Steam power

Pressure and ability to get a 300ml milk quantity in a 0.6l pitcher spinning. The more a pitcher needs to be tilted, the less powerful the machine.

• Pitcher vertical (0°) → Strong power
• Slightly tilted → Good power
• Very tilted → Weak power
• No rotation → Unsatisfactory

B. Speed

Time needed to heat 300ml milk from 8°C (refrigerator temperature) to 60°C. In this scenario, the milk is deliberately not frothed but only heated (steam wand inserted deep).

C. Workflow & Ergonomics

• No-burn steam wand or insulated design
• Ball swivel joint (360° freely movable)
• Rotary valve (precise steam volume control) vs. simple toggle switch

D. Condensation

How much condensation or residual moisture does the machine expel before steaming begins? Too much condensation dilutes the milk.

The robustness test

We don't just test whether volumetrics are present, but how robustly they work under changing conditions.

The 5-stage stress test procedure

Setup:

• Coffee: Apas Espresso (medium roast)
• Grinder: Mahlkönig E64 GbW
• Brew ratio: 1:2.5
• Target size: Depending on basket size, stays constantly set

The 5 stages:

1. Hardware baseline (5×): Scace basket (simulated resistance) – testing pure technology without coffee puck influence
2. Reference espresso (5×): Real conditions with standard grind
3. Profile check (5×): Test with complex profiles (e.g., pre-infusion) to check whether logic remains stable with pressure fluctuations
4. Stress test "coarser" (3×): Grind set one full notch coarser (faster flow)
5. Stress test "finer" (3×): Grind set one full notch finer (slower flow)

Goal: Does the machine recognize the changed flow (flowmeter) or does it stubbornly stop after time?

Evaluation grid (0-10 points)

Base points (Technology type):

• 0 pts: Manual (no volumetrics)
• 3 pts: Pure time control

Precision points (based on average deviation):

• 4 pts: ±4g deviation
• 6 pts: ±2g deviation
• 8 pts: ±0.5g deviation (scale level)

Bonus (+2 points max):

• +1 pt: Exact UI input (grams/milliliters instead of impulses)
• +1 pt: Integrated scale

With scale coupling, we run the protocol twice: once without, once with scale.

This category evaluates how pleasant the machine is to use daily. A technically brilliant machine that annoys you every day loses points here.

Point distribution (max. 10 points)

• Interface & control (max. 3 points + 0.5 bonus)
• Daily routine (max. 3 points)
• Cleaning & maintenance (max. 3 points)
• Documentation (max. 1 point)

A. Interface & Control (0-3.5 points)

For machines WITH display/software:

• 3.0 pts: Intuitive, no manual needed, quick response time (<0.5s), good readability
• 2.5 pts: Good, short learning curve, display readable in sunlight
• 2.0 pts: Solid, occasional manual reference needed, acceptable readability
• 1.5 pts: Cumbersome menu navigation, frequent manual lookup needed
• 1.0 pts: Confusing, sluggish response, poor readability
• 0.5 pts: Frustrating to use

For machines WITHOUT display (mechanical/analog):

• 3.0 pts: High-quality control elements, clear pressure point, precise rotary controls
• 2.0 pts: Solid haptics, functional
• 1.0 pts: Wobbly feel, imprecise adjustment
• 0 pts: Cheap plastic, excessive play

Bonus: +0.5 pts for programmable shortcuts/function buttons

B. Daily routine (0-3 points)

Each met criterion gives 0.5 points (max. 3):

• ✓ Water tank easy to remove/fill without binding
• ✓ Water level always clearly visible or last shot technology
• ✓ Drip tray sufficiently dimensioned (>500ml) and easy to remove
• ✓ Basket wrench smooth, secure latching without force needed
• ✓ Integrated water filter in tank present
• ✓ Ergonomic arrangement of all daily-use elements

C. Cleaning & Maintenance (0-3 points)

• 3.0 pts: Automatic cleaning programs present AND shower easily removable with spoon or included tool
• 2.5 pts: Automatic programs present, partial tool-free disassembly, tool not in package
• 2.0 pts: Manual cleaning simple, clear machine prompts for maintenance cycles
• 1.5 pts: Manual cleaning possible, somewhat fidgety
• 1.0 pts: Cumbersome disassembly, unclear maintenance hints
• 0.5 pts: Hard-to-reach parts, no maintenance hints

D. Documentation (0-1 point)

• 1.0 pts: Excellent manual (clearly structured, good imagery, complete)
• 0.5 pts: Acceptable manual with minor gaps
• 0 pts: Poor or unclear manual

Penalty: -1 point for significant "annoyance factors" (excessively loud alert tones, unnecessarily complex procedures, unexplained long wait times)

The technical foundation

Why temperature is so critical

Brew temperature is one of the most critical parameters in espresso. Just 2°C difference can drastically change the taste: too cold = sour and under-extracted, too hot = bitter and burnt. We don't rely on manufacturer specifications but measure ourselves – under standardized conditions.

Our measurement setup

Temperature measurement:

• Device: Scace 2 with data logger
• Calibration: Before each test series with boiling water
• Measurement: Directly in the basket

Power measurement:

• Device: Christ CLM1000 Professional (Plus)
• Resolution: 0.1 Wh (0.0001 kWh), second-by-second readings
• Logging: Saved as CSV file

Environmental conditions:

• Water temperature in tank: 20°C (±2°C)
• Target temperature: 93°C

The 93-degree standard and offset calibration

Our reference value is a brew temperature of 93°C.

With offset: If the machine has adjustable temperature control (PID), we calibrate the offset so that the machine in stable state (shots 3-5) delivers approximately 93°C at mean value.

Without offset: If temperature is not adjustable, we measure the machine in its factory setting.

When is a machine considered "heated up"?

Manufacturer claims are often optimistic ("Ready after 5 minutes"). We define the status "heated up" more strictly:

Example: If the machine settles at 92.5°C after an hour, it must deliver at least 91.5°C on the first shot after heat-up time.

Special case: With light baskets (plastic/silicone insert) we allow 1.5 degree deviation since they work thermally different than our measurement basket.

We determine actual heat-up time in a cold-start procedure in 5-minute intervals (5, 10, 15, 20 min) until this criterion is met.

The two test protocols

The KM Protocol – The Daily Use Test

Simulates typical home use: After heat-up time we extract 5 double espressos at one-minute intervals.

Procedure:

1. Clamp Scace in place
2. Start shot: extract 27 seconds
3. Dwell time: keep Scace clamped for 15 seconds
4. Unclamp
5. Rest: let Scace sit unclamped for 45 seconds
6. Flush: 2-second flush
7. Repeat (total 5 shots)

Goal: Check if the first shot is solid (cold-start performance) and whether the machine maintains temperature over a series.

The WBC Protocol – The Stress Test

Per World Barista Championship specs, the machine must perform under continuous fire: 14 shots in quick intervals plus simultaneous milk steaming. This shows whether the machine has reserves for guests or café use.

The WBC protocol is performed after one hour of runtime.

The 3-cluster scoring model (max. 10 points)

Cluster A: KM Daily Usability (max. 5.3 points)

A1. Cold-start behavior / First shot (max. 3.3 points)

Comparison: Mean value shot 1 vs. mean value shots 3-5 (steady state)

• Difference < 0.5°C → 3.3 points
• Difference < 1.0°C → 2.7 points
• Difference < 1.5°C → 2.0 points
• Difference < 2.5°C → 0.7 points
• Difference > 2.5°C → 0.0 points

A2. Consistency & Calibration (max. 2.0 points) – FAIRNESS UPDATE

Testing the stability of shots 3, 4, and 5 against each other.

Scenario A (Stable): Delta of mean values < 0.5°C?
→ 2.0 points (Full points for technical precision, regardless of absolute value)
→ Requirement: Note offset correction in report

Scenario B (Unstable): Delta > 0.5°C?
→ Evaluation based on deviation from target (93°C)

• ±0.5°C → 2.0 points
• ±1.5°C → 1.3 points
• >1.5°C → 0.0 points

Cluster B: Intra-shot stability (max. 2.7 points)

What happens during the 25 seconds of extraction?

Time-to-temperature (TTT):
Must reach target corridor (mean ±2°C) by second 10 at the latest.
If later ("Late Riser"): Automatically max. 0.7 points

Curve shape:

• Flatline: Horizontal (Δ < 0.5°C) → 2.7 points (Technical ideal)
• Drop: Slight decrease (< 2.0°C) → 2.0 points (Sensorically good – prevents bitterness)
• Rise: Slight increase (< 2.0°C) → 1.3 points (Sensorically critical – aggressive extraction)
• Drift/Unstable: Change > 2.0°C → 0.7 points
• Chaos: → 0.0 points

Cluster C: WBC Performance (max. 2.0 points)

Does the machine hold up at the garden party too?

We fire 14 espressos in quick succession.

Evaluation:

• Pro level: Window < 1°C (max-min) → 2.0 points
• Household level: Window < 3°C (max-min) → 0.7 points
• Failure: Window > 3°C → 0.0 points

Evaluation grid for heat-up time (evaluated separately as its own category)

The evaluation goes far beyond first impressions. We conduct a systematic inspection.

The categories (max. 10 points)

A. Surface quality

• Visual inspection of all housing parts (paint, polish, coating)
• Inspection for uniform gap dimensions (< 1mm = very good, > 1.5mm = problematic)
• Tactile inspection of all edges for sharpness or burrs
• Fit of attached parts (drip tray, tank cap, cup rail)

B. Material & Haptics

• Identification of primary materials used (metal, plastic, wood)
• Assessment of material thickness (e.g., sheet metal thickness >1.5mm = high-quality)
• Tactile inspection of controls (pressure point, sturdiness, play)

C. Construction & Stability

• Assessment of housing torsional stiffness
• Inspection of stability
• Quality of visible fasteners and mechanical connections

D. Interior quality (if visible)

• Cleanliness of cable/hose routing as indicator of manufacturing care

Note on durability

We currently do not fully test the actual durability of an espresso machine. We simply cannot do that responsibly for multiple machines. We therefore refer to reports from people who have purchased an espresso machine. The sentiment online gives a good overview of whether an espresso machine is prone to failure.

If we do hear from our own use or from experiences with our Home Barista course participants about increased problems with test machines, we mention that in our test reports.

Noise level is a decisive comfort factor, especially in open kitchen-living areas. We don't rely on subjective hearing but measure sound pressure level under standardized conditions.

Test scenario:

• Distance: 20 cm at brew group height
• Environment: Film studio (controlled acoustic conditions)
• Measured: Maximum value during an espresso shot under load (against blind basket or coffee puck at 9 bar)

Evaluation grid (0-10 points)

Subjective correction: In extreme cases, the grade can be adjusted by max. 1 point if the type of noise is particularly disturbing (high-frequency whistling, rattling) or particularly pleasant (sonorous, deep hum). This must be explicitly justified in the test report.

We don't evaluate the amount of plastic waste but the quality of the tools. A perfectly fitting tamper is more valuable than five poorly made screens.

Scoring system based on point collection (max. 10 points)

Base equipment (required elements)

• 2-spout basket present: +1 pt
• At least 2 screens (different sizes): +1 pt
• Tamper included: +1 pt
• Blind basket for backflushing: +0.5 pt

Quality of core components

• Tamper perfect fit (<1mm play in basket): +2 pts
• Tamper good fit (1-2mm play): +1 pt
• Basket made of stainless steel: +0.5 pt
• Screens burr-free, high-quality stamping: +1.5 pts
• Screens acceptable: +0.5 pt
• Basket high-quality (metal, good haptics): +1 pt

Additional accessories

• 1-spout basket: +0.5 pt
• Bottomless basket: +1 pt
• Milk pitcher (only if high-quality): +0.5 pt
• Cleaning supplies/cleaning brush: +0.5 pt
• Water filter stick: +0.5 pt
• Cups: +0.5 pt
• Useful extras (leveler, WDT tool, apron, etc.): +0.5 pts per item (max. +1 pt total)

Penalty

• Tamper too small (>2mm play): -1 pt
• Burr formation on screens: -0.5 pt
• Non-functional cheap accessories: -0.5 pt

Maximum: 10 points (excess is capped)

We evaluate the machine's performance capability in continuous operation ("stress test"). For most home baristas less relevant, but important if you frequently entertain guests.

Point distribution (max. 10 points)

A. Temperature stability under sustained load (0-4 points)

Result from WBC protocol (Chapter 6):

• 4 pts: WBC protocol passed with window <1°C
• 3 pts: WBC protocol passed with window <3°C
• 2 pts: Slight instability but still usable
• 1 pt: Clear performance loss
• 0 pts: Failure under load

B. Steam volume potential (0-3 points)

Test: 5 milk steaming cycles (300ml each) consecutively in 90-second intervals

• 3 pts: All 5 cycles without performance loss, consistent steam power
• 2 pts: Slight performance loss from cycle 4-5 onwards but still usable
• 1 pt: Clear performance loss from cycle 3 onwards
• 0 pts: Machine can't handle 5 cycles

C. Capacities (0-3 points)

• +1.5 pts: Direct water connection
• +1 pt: Drip tray with drain
• +0.5 pts: Large cup shelf with pre-heating function (>6 cups)

The coffee maker energy score

Energy efficiency in modern espresso machines is no longer a niche topic but a central quality feature. To create fair comparability between a small thermoblock and a large dual boiler, we developed a differentiated evaluation system. Further explanations of the measurement system available here. Our power protocol is now being used by various espresso machine manufacturers in development.

Our measurement setup

Device: Christ CLM1000 Professional (Plus)

• Resolution: 0.1 Wh (0.0001 kWh)
• Logging: Second-by-second readings as CSV file

Starting conditions:

• Machine at rest for at least 6 hours
• Water tank filled to maximum with room temperature water (20-24°C)
• Cold start: Machine unplugged, then plugged in and switched on

The complete measurement protocol

We conduct a comprehensive measurement protocol on each machine to cover all usage scenarios:

The evaluation principle: "Espresso first"

Although we run the complete protocol, we evaluate the machine based on the typical daily-use scenario:

The principle of best possible efficiency applies: If a machine (e.g., a dual boiler) has the function to shut down the energy-hungry steam boiler, we evaluate it in this economical "espresso mode". We want to reward technological progress and energy-saving features.

Transparency: Additional measurement values in our tests

In our detailed test reports, we additionally show:

For classic two-circuit machines or machines without shut-down function for the steam boiler, the espresso value is identical to the cappuccino value (minus pure steam draw) – which correspondingly results in a lower score.

Evaluation scale (1-10 points)

The scale is based on current technology standards and uses the value "heat-up + 1 double espresso":

The fair value formula

Evaluation of the price-to-performance ratio (PPR) is not subjective but based on a mathematical "fair value" model. This sets the objectively measured overall quality of the machine in relation to its current market price.

Why a formula?

A 3,000-euro machine doesn't need to be three times as good as a 1,000-euro one to have good PPR. This is due to the economic law of diminishing marginal utility: Each additional euro spent yields less additional quality. A perfect 10/10 machine for 5,000 euros can have worse PPR than a solid 7/10 machine for 800 euros.

The calculation model

Step 1: Determine quality basis (0-100 scale)

We take the weighted average of all technical and tactile performance categories (chapters 1-10, WITHOUT the PPR category itself) and scale it to 0-100:

Step 2: Calculate raw value (square root correction)

To not penalize expensive high-end machines linearly, the price is "de-escalated" via square root:

Example:

• Machine A: Quality basis 80, price 1,600 EUR
  → Raw value = (80 / √1600) × 10 = (80 / 40) × 10 = 20.0
• Machine B: Quality basis 70, price 900 EUR
  → Raw value = (70 / √900) × 10 = (70 / 30) × 10 = 23.3

→ Machine B has better PPR despite lower quality!

Step 3: Scale to 0-10 grade (fair value 2025)

The raw value is converted to a grade based on current market data:

The anchor (grade 5.0): A raw value of 15.0 defines the current market standard for a fair price (e.g., solid mid-range dual boiler).

The slope: A change in raw value of 3.0 points leads to a change in grade of 1.0.

Formula:

Limit: Min. 0 points, max. 10 points

Interpretation

• Grade > 8 Price bargain. Above-average performance for the money
• Grade 5 – 6.5 Fair value. Price matches offered performance
• Grade < 5 Enthusiast premium. You pay a disproportionate premium for nuances, design, or brand

How to interpret our test results

1. The overall score is just the beginning

A 7.5 means "Very Good" – but maybe the machine is weak in categories you don't care about and excellent in categories you do care about. Always check the individual scores!

2. Only compare within similar price ranges

A machine for 800 EUR with score 7.5 is a different caliber than one for 2,500 EUR with score 7.5. Use the price-to-performance grade for fair comparisons.

3. Identify your "must-haves"

Ask yourself:

• Do I drink mainly espresso or milk drinks? (→ Espresso potential vs. steaming quality)
• Do I use the machine daily or weekends? (→ Heat-up time, usability)
• Is sustainability important to me? (→ Power consumption)
• Do I frequently entertain guests? (→ Volume potential)

4. Use the individual evaluations as a checklist

Each category has sub-points. If you know, for example, that smart profiling is important to you, look directly at Chapter 1 (Espresso potential) to see if the machine offers that.

5. Wait for personalized weighting (coming soon)

Transparency and continuous improvement

This test protocol is a living document. We continuously adapt it based on your feedback and our experience. Version 2.3 is already a significant development compared to our initial tests.

What we haven't fully tested yet

The exact point distribution in the categories steaming quality and build quality will be added once we've collected sufficient comparison data. However, the categories and evaluation criteria are already defined.

What you can do

• Give us feedback
• Tell us which aspects matter most to you
• Share your experiences with tested machines

Our promise

We stay objective, measurable, and transparent. Every machine is tested according to the same protocol. Every machine is purchased by us or in rare cases, with very expensive espresso machines, rented by us with a rental fee.

We take our time. Good testing takes time.

Closing remarks

A good test protocol is like a good recipe: It standardizes the process but allows room for nuance. We hope that with this guide, you not only better understand our tests but can also make your own purchasing decision on a solid basis.

In the end, only one thing matters: The espresso in your cup. All measurements, scores, and formulas are only a means to an end – to help you find the machine that fits your life, your needs, and your budget.

Good luck with your machine search!

Andrea Perin, Michel Indelicato, Benjamin Hohlmann