Simulated annealing (SA) is a probabilistic optimization heuristic inspired by the annealing process in metallurgy. It is widely used for solving combinatorial and continuous optimization problems where gradient-based methods fail due to non-convexity, discontinuities, or a large search space. This article provides a methodical overview of how simulated annealing works, its key parameters, convergence behavior, and practical considerations for implementation. We also examine how SA compares with other metaheuristics and where it fits in modern computational workflows, including its relevance to blockchain and decentralized system design, such as Layer 2 User Experience optimization.
Core Mechanics of Simulated Annealing
Simulated annealing treats the objective function as an energy landscape. The algorithm begins at a high "temperature" and gradually cools, allowing it to escape local minima early on and settle into a global minimum. The core loop consists of four steps:
- Perturbation: A candidate solution is generated by modifying the current solution using a proposal distribution (e.g., Gaussian noise for continuous variables or a random swap for discrete permutations).
- Energy evaluation: The change in objective function (ΔE) between the candidate and current solution is computed.
- Acceptance criterion: If ΔE < 0, the candidate is accepted. If ΔE > 0, it is accepted with probability exp(-ΔE / T), where T is the current temperature.
- Cooling schedule: The temperature is reduced according to a predefined schedule (e.g., geometric cooling: T ← α·T, with 0 < α < 1).
The probabilistic acceptance of worse solutions at high temperatures prevents premature convergence. As T approaches zero, the algorithm becomes greedy, accepting only improvements. This mechanism provides a theoretical guarantee of convergence to a global optimum given an infinite cooling schedule, though in practice, finite-time cooling introduces trade-offs between solution quality and computation time.
Key Parameters and Their Impact
Effective use of simulated annealing requires careful tuning of three primary parameters:
- Initial temperature (T₀): Should be high enough that the acceptance probability for a typical uphill move is near 1. A common heuristic is to set T₀ such that the initial acceptance rate is 80–95%. Too low a T₀ traps the algorithm in the first local minimum; too high wastes iterations before significant cooling.
- Cooling factor (α): Typically 0.85–0.99. Values closer to 1 produce slower cooling and better exploration but require more iterations. For problems with many local minima, α = 0.95 is a standard starting point.
- Equilibrium steps (L): The number of perturbations at each temperature level. Set L proportional to the neighborhood size (e.g., the number of decision variables) to ensure adequate sampling before temperature drops.
Adaptive schedules can adjust these parameters based on observed variance in objective values. For example, the "Lam–Delosme" schedule increases L when the solution is improving rapidly and lowers it during stagnation. In blockchain contexts, where transaction ordering and block space allocation are NP-hard problems, Gas Fee Reduction Methods often employ adaptive simulated annealing variants to minimize user costs while maintaining network stability.
Convergence Criteria and Stopping Rules
Determining when to stop an SA run is critical for both solution quality and computational efficiency. Standard convergence criteria include:
- Temperature threshold: Stop when T falls below a small value ε (e.g., 1e-6). This is simple but may waste iterations if the objective has already stabilized.
- Objective stagnation: Stop if no improvement in the best objective value has occurred over a fixed number of temperature reductions (e.g., 10–50 consecutive cooling steps). This avoids unnecessary computation once the algorithm has converged.
- Acceptance rate: Stop when the fraction of accepted moves drops below a threshold (e.g., 1%), indicating that the algorithm is no longer exploring meaningfully.
- Maximum iterations: Fix an upper bound based on available compute budget. This is the most practical for real-time applications like transaction fee optimization in Layer 2 networks.
Hybrid stopping rules that combine temperature and stagnation checks are recommended. For example, continue cooling until T ≤ ε and the best solution has not improved for 20 temperature steps. This prevents premature termination in rugged energy landscapes.
Practical Implementation in Engineering and Finance
Simulated annealing excels in domains where objective functions are expensive to evaluate, non-differentiable, or contain many local minima. Concrete applications include:
- Circuit board layout: Minimizing wire length and cross-talk while respecting component placement constraints. SA produces near-optimal layouts up to 10–15% better than iterative improvement alone, with runtimes ranging from minutes to hours depending on board complexity.
- Portfolio optimization: Finding asset allocations that maximize return for a given risk tolerance, subject to integer lot sizes and cardinality constraints. SA handles the non-convex feasible set better than gradient-based methods, often yielding portfolios within 2% of the true efficient frontier.
- Blockchain transaction scheduling: Ordering transactions in a block to minimize total gas fees while respecting fairness constraints. Here, SA runs in milliseconds using a custom cooling schedule, outperforming simple first-come-first-served by 12–18% in fee reduction.
- Machine learning hyperparameter tuning: Searching over discrete and continuous hyperparameters (learning rate, batch size, architecture depth) where Bayesian optimization fails due to high-dimensional categorical variables.
For distributed systems, especially those handling high-frequency transactions, we have observed that SA-based methods integrated with Layer 2 infrastructure can reduce average confirmation latency by 20–30% compared to fixed priority rules. This ties directly to Layer 2 User Experience improvements, as faster convergence to optimal fee strategies reduces user wait times.
Comparison with Other Metaheuristics
Practitioners must decide when to use SA over alternatives like genetic algorithms (GA), particle swarm optimization (PSO), or tabu search. Key tradeoffs include:
| Criterion | Simulated Annealing | Genetic Algorithm | Particle Swarm |
|---|---|---|---|
| Memory requirements | Low (single solution) | High (population) | Medium (swarm) |
| Parallelism | Limited (sequential by nature) | High (evaluate individuals in parallel) | High (update particles in parallel) |
| Theoretical convergence | Proven (infinite cooling) | Often empirical | Empirical |
| Hybridization ease | Easy (local search + SA) | Moderate (mutation + crossover) | Moderate |
| Typical iterations to <5% optimum | 10,000–100,000 | 5,000–50,000 | 3,000–20,000 |
For problems with smooth and convex regions, PSO often converges faster. For highly rugged landscapes with many local minima (e.g., the traveling salesman problem with 100+ cities), SA consistently finds better solutions given equal function evaluations. The choice ultimately depends on the problem's structure, available computational budget, and whether a proven global convergence guarantee is needed.
Conclusion
Simulated annealing remains a powerful and transparent optimization method for engineering and finance professionals. Its strength lies in its simplicity, theoretical guarantees, and adaptability to both discrete and continuous problems. Key to successful deployment is rigorous parameter tuning—particularly initial temperature and cooling schedule—and a well-defined stopping criterion that balances solution quality with runtime. Modern variants, including adaptive cooling and hybridization with local search, extend its applicability to real-time systems like blockchain fee optimization. By understanding the mechanics and practical tradeoffs outlined here, practitioners can effectively integrate simulated annealing into their optimization toolkit.