Normal modes of a waveguide as eigenvectors of a self-adjoint operator pencil

A waveguide with a constant, simply connected section S is considered under the condition that the substance filling the waveguide is characterized by permittivity and permeability that vary smoothly over the section S, but are constant along the waveguide axis. Ideal conductivity conditions are assumed on the walls of the waveguide. On the basis of the previously found representation of the electromagnetic field in such a waveguide using 4 scalar functions, namely, two electric and two magnetic potentials, Maxwell’s equations are rewritten with respect to the potentials and longitudinal components of the field. It appears possible to exclude potentials from this system and arrive at a pair of integro-differential equations for longitudinal components alone that split into two uncoupled wave equations in the optically homogeneous case. In an optically inhomogeneous case, this approach reduces the problem of finding the normal modes of a waveguide to studying the spectrum of a quadratic self-adjoint operator pencil.


Introduction
Consider a waveguide representing a cylinder of constant cross-section filled with an optically inhomogeneous substance, which we will characterize with a permittivity and a permeability. Hereinafter, we will make use of a Cartesian coordinate system, the -axis of which coincides with the waveguide axis. We will assume that the permittivity and permeability do not depend on , but are piecewise smooth functions of , . The normal modes of a waveguide are non-trivial solutions of Maxwell's equations of the form satisfying the conditions of ideal conductivity of the waveguide walls. Here the positive parameter is the circular frequency of the wave, = / is the wave number, and the complex parameter is the phase constant.
Substitution of the expression (1) into Maxwell's equations yields 8 equations for 6 unknowns , … , , containing two parameters, and . It is usually assumed that the wavenumber of the considered waves is given, and then we get an eigenvalue problem with respect to the spectral parameter . This problem was successfully solved in the case of constant and , thanks to the introduction of two scalar potentials, the electric and magnetic Borgnis functions and [1], [2]. In the attempt to study a general case undertaken in the beginning of 2000s [3]- [5], it was not possible to introduce potentials and the problem was investigated with respect to three randomly chosen field components. With this approach, the normal waves of the waveguide turned out to be eigenfunctions of some non-self-adjoint quadratic operator pencil acting in a space specially selected by the functional.
Not all properties of a hollow waveguide can be extended to the case of a waveguide filled with an optically inhomogeneous substance. We can confidently reject the hypothesis of the field decomposition into TE-and TM-waves, since the existence of hybrid modes has been proved analytically in half-filled waveguides [6, § 3.5]. With less confidence, one can reject the hypothesis that the propagation constants of normal modes cannot have both real and imaginary parts. In a series of numerical experiments [7]- [9], it was shown that the propagation constants of the normal modes of an axially symmetric waveguide with a dielectric core can leave the real and imaginary axes of the complex plane. However, to calculate these eigenvalues, we used the truncation method and standard solvers to find the eigenvalues of non-self-adjoint matrices. Our experiments in FreeFem++ [10] showed that solvers of this kind can introduce a complex addition to the spectrum of a self-adjoint problem.
We have recently succeeded in extending the theory of Borgnis functions to the case of a waveguide filled with optically inhomogeneous matter [11], [12]. In this case, we have increased the number of potentials to four. Mode hybridization makes one think that the system of equations for the potentials does not split in the general case, but we cannot exclude the fact that this system is written in a self-adjoint form. In this paper, we intend to present such a self-adjoint formulation of the problem of finding the normal modes of a waveguide.

Representation of the electromagnetic field using electric and magnetic potentials
Let for simplicity the waveguide cross section be a planar simply connected domain with smooth boundary , and let the permittivity and permeability be smooth functions of , . Denote as , the segments of finite of infinite length on the axes and , respectively and assume = in all cases except = 1 . The unit external normal vector to the curve will be denoted as ⃗ = ( , , 0) , and the tangent vector in the -plane as ⃗ = (− , , 0) . Also for brevity let us assume that As in the theory of Borgnis functions, the scalar function turning into zero at the boundary will be referred to as electric potential and the scalar function satisfying the Newmann condition = 0 at the boundary -as magnetic potential × × . Hereinafter electric and magnetic potentials are denoted by and , respectively, with different indices.
The main result about the four potentials established by us earlier [12] is that the electromagnetic field allows a representation in terms of four potentials, namely, two electric potentials , ℎ and two magnetic ones , ℎ : Therefore, below we seek the solution of Maxwell's equations in a waveguide in the form (2) without any loss of generality.

Maxwell's equations in terms of potentials
Substituting expression (2) into Maxwell's equations, we get 8 rather than 6 independent equations. Four of these equations allow expressing the potentials in terms of the longitudinal field components и . The relation is determined by classical boundary-value problems. The electric potentials can be found as solutions of Dirichlet problems and The magnetic potentials can be found as solutions of Newmann problems and In this case from Maxwell's equations it follows that In problems of monochromatic wave propagation is equivalent to multiplication by the number , therefore from (7) the solvability of the above problems with the Neumann conditions follows.
The rest two equations can be written in the form Substituting here the expressions for the potentials in terms of the field longitudinal components and , which are obtained by solving the problems (3)-(6), we rewrite this system in the form where as an unknown we consider ⃗ = ( , ) , composed of the field longitudinal components. Herê, … ,̂ℎ are symmetric positively defined integral operators acting in 2 ( ), and̂is a non-symmetric integral operator. This operator makes impossible the separation of the problem into two independent problems, due to which the hybridization of modes occurs. We will call it a hybridization operator.

Normal modes of a waveguide
Normal mode (1) corresponds to a solution of the system (9) in the form Taking the dependence on , into account, we can formulate the problem of finding the normal modes of the waveguide as an eigenvalue problem with respect to the spectral parameter . Thus the problem of finding normal modes reduces to the analysis of the spectrum of the polynomial operator pencil where the coefficientŝ0,̂1,̂2 are self-adjoint operators with respect to a scalar product in 2 ( ) × 2 ( ); the senior coefficient̂2 is a positively defined and completely continuous operator; the operator̂1 is completely continuous and the operator̂0 is bounded and reversible. The pencils of such form arouse in the linear theory of small damped oscillations and were studied by M. G. Krein and G. K. Langer [13, §12].

Conclusion
By introducing four potentials, we were able to reduce the problem of wave propagation in a waveguide filled with an inhomogeneous substance to a linear second-order partial differential equation (9), the coefficients of which are self-adjoint operators. In this case, the problem of finding normal waves is reduced to studying the spectrum of the quadratic operator pencil (11). Thus, the formulation of the eigenvalue problem retains the symmetry characteristic of scalar eigenvalue problems.
This means, first of all, that with discretization by the truncation method, we obtain a problem for the eigenvalues of a quadratic self-adjoint matrix pencil. By means of the known procedure [13, §12] it can be reduced to the generalized eigenvalue problem̂=̂, where,̂are self-adjoint matrices. This opens up possibilities for using specialized eigenvalue solvers.