Bayesian Optimization for Nonlinear System Identification and Pre-Distortion in Cognitive Transmitters

We present a digital signal processing (DSP) scheme that performs hyperparameter tuning (HT) via Bayesian optimization (BO) to autonomously optimize memory tap distribution of Volterra series and adapt parameters used in the synthetization pre-distortion (DPD) filter for optical transmitters. Besides providing time-efficient technique, this work demonstrates self-adaptation DPD hyperparameters correct component-induced nonlinear distortions as different driver amplifier (DA) gains, symbol rates modulation formats are used, leads an improvement transmitter performance. The has been validated back-to-back (b2b) experiments using dual-polarization (DP) 64 256 quadrature amplitude (QAM) formats, 80 GBd. For DP-64QAM at GBd, it is shown reduces required signal-to-noise ratio (OSNR) bit error 10 -2 by 0.9 dB 0.6 with respect linear heuristic approach, respectively. Moreover, we show proposed approach also complexity 75% conjunction use polynomials (MP), while achieving similar performance filters.