The Hamilton–Jacobi Equation: The Forgotten Theory of Wave-Particle Unification

——Quantum Mysteries Imbedded in Classical Mechanics All Along

Abstract

A startling fact: the Hamilton–Jacobi Equation (HJE) had fully characterized the essence of wave-particle duality long before the birth of quantum mechanics. Through an in-depth analysis of the physical connotation of the action S, we find that the so-called "quantum revolution" of the 20th century was, in essence, a rediscovery and formalization of 19th-century classical theory. Within the framework of Natural Quantum Theory (NQT), quantum mechanics is not an independent new physical system, but a natural extension of Hamilton–Jacobi theory in the frequency domain.

I. Misreading of History: A Reunion Delayed by Ninety Years

1.1 The Forgotten Prophecy

In 1834, William Hamilton wrote in his epoch-making paper: "There exists a profound analogy between optics and mechanics—so much so that mechanics may even be regarded as a branch of optics." This was not a poetic metaphor, but a precise physical insight. He astutely observed that the mechanical equations describing particle trajectories and the geometric optics equations describing light propagation are entirely isomorphic in their mathematical structure.

Subsequently, Carl Jacobi systematized this idea and constructed a complete mechanical system centered on the action S. Its core equation—the Hamilton–Jacobi Equation:

\(\frac{\partial S}{\partial t} + H\left(q_1, \dots, q_n, \frac{\partial S}{\partial q_1}, \dots, \frac{\partial S}{\partial q_n}, t\right) = 0\)

not only describes particle motion but also reveals a universal law of wave propagation.

1.2 de Broglie’s "Rediscovery"

In 1924, de Broglie proposed the matter wave hypothesis, linking particle momentum p to wavelength \(\lambda\):

\(\lambda = \frac{h}{p}\)

However, a return to Hamilton–Jacobi theory reveals that this relationship was already implicit within it. In this framework, momentum is given by the gradient of the action:

\(p = \nabla S\)

and the equipotential surfaces of \(S = \text{const}\) are the wavefronts. The distance between adjacent wavefronts corresponds exactly to the wavelength; when the change in action equals the Planck constant h, the de Broglie relation emerges naturally—no new assumptions are required.

II. The Dual Nature of the Action: Geometric Unification of Waves and Particles

2.1 Deep Isomorphism Between Optics and Mechanics

In geometric optics, Fermat’s principle states that light travels along the path that extremizes the optical path (i.e., the action). The corresponding eikonal equation is:

\((\nabla S)^2 = n^2(x, y, z)\)

where S is the eikonal function and n is the refractive index.

In classical mechanics, the Hamilton–Jacobi Equation for a free particle is written as:

\((\nabla S)^2 = 2m\left[E - V(x, y, z)\right]\)

The two equations are identical in mathematical form, with \(E - V\) playing the role of a "mechanical refractive index."

Key Insight: This is not an analogy, but the manifestation of the same physical principle in different domains—the action field \(S(x,t)\) is the underlying entity that unifies waves and particles.

2.2 Geometric Relationship Between Wavefronts and Trajectories

The action S defines a family of equipotential surfaces in spacetime. These surfaces admit a dual interpretation:

• Wave perspective: Surfaces of constant S = wavefronts

• Particle perspective: Particle trajectories are always perpendicular to wavefronts

This relationship can be rigorously derived from Hamilton’s equations. Let \(p = \nabla S\), then the particle velocity is:

\(v = \frac{dq}{dt} = \frac{\partial H}{\partial p} \propto \nabla S\)

Thus, trajectories are orthogonal to surfaces of \(S = \text{const}\) everywhere—this is the geometric cornerstone of wave-particle unification.

2.3 Physical Reality of the Action

S is not an abstract mathematical tool, but an observable physical quantity:

• Double-slit interference: Bright and dark fringes are determined by the action difference between the two paths:\(\Delta S = S_1 - S_2 =  \begin{cases}  n\pi\hbar & \text{(bright fringes)} \\  \left(n + \frac{1}{2}\right)\pi\hbar & \text{(dark fringes)}  \end{cases}\)

• Aharonov–Bohm effect: Even in regions where the electromagnetic field is zero, the vector potential A still produces an observable phase shift by altering the action:\(\Delta S = e\oint A \cdot dl\)

This confirms that the action has an ontological status independent of field strength.

III. From Hamilton–Jacobi to Schrödinger: The Inevitable Path of Complexification

3.1 Classical Roots of the Schrödinger Equation

While the Schrödinger equation appears to introduce entirely new physics, it is in fact a complete extension of Hamilton–Jacobi theory. Let the wave function be:

\(\psi = R e^{iS/\hbar}\)

where R is the amplitude and S is the phase (i.e., the action). Substituting this into the Schrödinger equation:

\(i\hbar \frac{\partial \psi}{\partial t} = \hat{H} \psi\)

Separating the real and imaginary parts yields two equations:

• Real part (modified HJE):\(\frac{\partial S}{\partial t} + \frac{(\nabla S)^2}{2m} + V - \underbrace{\frac{\hbar^2}{2m} \frac{\nabla^2 R}{R}}_{\text{Quantum Potential } Q} = 0\)As \(\hbar \to 0\), the quantum potential vanishes, returning to the classical HJE.

• Imaginary part (continuity equation):\(\frac{\partial R^2}{\partial t} + \nabla \cdot \left(R^2 \frac{\nabla S}{m}\right) = 0\)This describes the conservation of probability current (or energy density).

3.2 The Nature of the Quantum Potential

The so-called "quantum potential":

\(Q = -\frac{\hbar^2}{2m} \frac{\nabla^2 R}{R}\)

is not a mysterious new interaction, but a self-diffusion effect caused by the spatial curvature of the wave packet—a natural consequence of wave behavior. Within the NQT framework, it arises from the nonlocal structure of the action field.

3.3 The True Meaning of the WKB Approximation

The WKB method is often regarded as a bridge "from quantum back to classical," but its deeper significance is precisely the opposite: it proves that the classical HJE itself contains quantum behavior.

In the WKB approximation:

\(\psi \approx A \exp\left(\frac{i}{\hbar} \int p \cdot dq\right) = A e^{iS/\hbar}\)

The quantization condition:

\(\oint p \cdot dq = \left(n + \frac{1}{2}\right)h\)

is essentially a requirement for the wave function to be single-valued—this is the standard constraint of classical wave theory on closed paths.

IV. Experimental Verification and Future Predictions

4.1 Classical Interpretation of Existing Phenomena

• Double-slit interference: Each particle travels along a definite trajectory but is guided by the global action field; interference patterns result from the coherent superposition of S along different paths.

• Quantum tunneling: Momentum becomes imaginary (\(p = i|p|\)) inside the potential barrier, and the action becomes complex, leading to exponential decay of the wave function—this is identical to the behavior of electromagnetic waves in forbidden bands.

4.2 Testable New Predictions

Based on a complete understanding of the HJE, we propose three verifiable predictions:

1. Determinism of subwavelength trajectoriesWeak measurements should reveal that single-particle trajectories are strictly perpendicular to surfaces of constant S, with curvature determined by \(\nabla^2 S\)—there is no randomness, only streamlines.

2. Direct detection of the actionReconstruct \(S(x, t)\) using geometric phases (gauge-invariant parts) or Stokes parameters to verify its continuity and single-valuedness.

3. Continuous quantum-classical transitionBy tuning mass or temperature, observe the smooth evolution of the de Broglie wavelength—there is no abrupt "quantum threshold."

V. Theoretical Reconstruction and Philosophical Clarification

5.1 Implications for the Interpretation of Quantum Mechanics

• Copenhagen interpretation: Wave function collapse is a redundant assumption; measurement is merely the selection of a specific projection of the action field.

• Many-worlds interpretation: No "cosmic splitting" is needed; all possibilities are already inherent in the S field.

• de Broglie–Bohm theory: Gains support—the pilot wave is \(\nabla S\), trajectories are streamlines, and nonlocality stems from the globality of S.

5.2 Refounding Quantum Field Theory

Quantum fields can be regarded as superpositions of action modes:

\(\phi(x, t) = \sum_n a_n e^{iS_n(x, t)/\hbar}\)

• Particles = topological solitons of the action

• Interactions = nonlinear couplings of the action

• Renormalization = restoration of the completeness of the short-wavelength action spectrum

5.3 Return to Ontology

We can advocate the following ontological hierarchy:

\(\text{Fundamental reality} = S(x, t) \Rightarrow \psi = e^{iS/\hbar} \Rightarrow \text{Particles} = \text{singularities of } S\)

From this:

• Causality is restored: The HJE is a deterministic equation

• Uncertainty is epistemological: Arising from the finite width of wave packets

• The measurement problem is naturally resolved: No conscious intervention is needed, and objectivity is guaranteed by action conservation

VI. Historical Reflection and Future Directions

6.1 Why It Was Long Ignored?

• Historical contingency: The success of the Schrödinger equation overshadowed its classical origins; the probabilistic interpretation gained dominance too early.

• Conceptual misunderstanding: "Quantum" was treated as entirely new physics, rather than the complete form of classical wave theory; the wave function was mistaken for the fundamental reality.

6.2 Unification of Classical and Quantum Physics

Within the HJE framework, physics exhibits a clear hierarchy:

\(\text{Classical Mechanics} \subset \text{Wave Mechanics} = \text{Quantum Mechanics}\)\((\text{Local limit}) \quad (\text{Complete theory}) \quad (\text{Frequency-domain formulation})\)

All interactions can be understood as manifestations of different symmetries of the action field:

• Gauge fields = connections of the action

• Gravity = geometric structure of the action

VII. Conclusion

The Hamilton–Jacobi Equation is not an approximation to quantum mechanics, but a complete theory encompassing all quantum phenomena. The "quantum revolution" of the 20th century was, in truth, a belated confirmation of physical truths already uncovered in the 19th century.

From the perspective of Natural Quantum Theory, this realization brings three key insights:

1. Quantum mechanics requires no new axioms—it is an inevitable extension of classical wave theory;

2. Wave-particle duality was already embedded in the geometric structure of the action S;

3. All "quantum mysteries" can be reduced to classical behaviors of the action field.

This is not a regression of theory, but a step toward a deeper, more unified physical picture. The action, not the wave function, is the primordial language of nature.

Nearly two centuries ago, Hamilton and Jacobi laid this path to unification for us. Today, it is time to brush off the dust of history and reembark on this forgotten yet future-oriented road.

"The fundamental principles of nature often appear in the simplest forms—we merely need sufficient wisdom to recognize them."