4 ECTS credits
110 h study time

Offer 1 with catalog number 4019801ENR for all students in the 2nd semester at a (E) Master - advanced level.

Semester
2nd semester
Enrollment based on exam contract
Impossible
Grading method
Grading (scale from 0 to 20)
Can retake in second session
Yes
Taught in
English
Partnership Agreement
Under interuniversity agreement for degree program
Faculty
Faculteit Ingenieurswetenschappen
Department
Electricity
Educational team
John Lataire (course titular)
Activities and contact hours
24 contact hours Lecture
24 contact hours Seminar, Exercises or Practicals
Course Content

General goal of the course

Engineers and scientists build models to understand, describe, predict and control the behaviour of complex dynamical processes. In order to construct these models it is necessary to extract the unknown parameters of these mathematical models from (noisy) measurements. In this course we first learn how to measure the characteristics of a dynamic system (Spectral analysis, frequency response function measurements). Next, we explain how experimental data can be turned into good mathematical models. Each concept is illustrated by a simple example. Besides a sound theoretical basis, we will provide the students also with hand on experience in the labs.

Outline:

Measurements:

  • A bird's eye view on instrumentation
    • What is a measurement? Measurement chain, the ideal instrument.
  • The Fourier based spectrum
    • Signal conditioning
    • The anti-alias filter
    • Discretisation in time/amplitude
    • Using the DFT for spectral analysis (Perfect reconstruction, leakage, definition of the bin)
    • Reducing the leakage via windowing
  • DFT-based network analysis
    • Properties of Linear Time-Invariant systems (Transfer function, Response to periodic excitations)
    • Transfer function estimation, periodic (multisine) excitations
    • Transfer function estimation, arbitrary excitations
      • Working with non-steady-state measurements
      • Transient suppression via non-rectangular windows
      • Improvement via spectral averaging
    • Noise influence, periodic excitations and arbitrary excitations
      • Bias and variance of estimates
    • Reducing the variability of a measured transfer function
      • different averaging approaches
  • Choice of the excitation signa
    • Sine wave, multisine, pulse, noise, PRBS excitation signals: pro's and cons

Data-driven modelling:

  • Introduction: three critical issues in data-driven modelling
  • Basic tools for analysing estimators
    • Stochastic convergence, law of large numbers, central limit theorem, Cramér-Rao lower bound
    • Stochastic properties: consistency, (asymptotic) bias, (asymptotic) covariance, (asymptotic) efficiency, (asymptotic) normality, robustness
  • Linear least squares
    • Noiseless regression matrix: stochastic properties, numerical stable calculation, weighted linear least squares
    • Noisy regression matrix: bias compensated least squares, (generalised) total least squares, instrumental variables method
    • Regularized linear least squares: L2 regularisation, Tikhonov regularisation
    • Outliers
  • Nonlinear least squares
    • Stochastic properties, weighted nonlinear least squares
    • Separable nonlinear least squares, variable projection method
    • Minimization algorithms: Newton-Raphson, Gauss-Newton, Levenberg-Marquardt, gradient descent, line search
    • Generation of starting values
  • Maximum likelihood method
    • Definition, asymptotic properties, invariance principle
    • Examples under standard and non-standard conditions
  • Bayesian approach
    • Definition, asymptotic properties, connection with maximum likelihood, trade-off data fit and prior knowledge
    • Bayesian linear regression
    • Gaussian process modelling: link with the Bayesian approach, link with regularised linear least squares, FIR estimation
  • Neural networks
    • Neural network models: feed forward, recurrent, and convolutional
    • Universal approximation theorem
    • Training via back propagation -- stochastic gradient descent
    • Early stopping
  • Tuning the model complexity
    • Use of validation data: training/validation data, leave-one-out-cross-validation, p-fold cross-validation
    • Use of penalty terms -- discrete tuning of the model complexity: final prediction error, Akaike information criterion, minimum description length
    • Use of penalty terms -- continuous tuning of the model complexity: L2 regularisation, L1 regularisation, elastic net

Hand-on-experience

  • Five labs will illustrate the different parts of the course with hands-on experiments and Matlab exercises.
Additional info

Prior knowledge

  • Basic skills in statistics (sample statistics, distributions, covariance matrix)
  • Basic skills in system theory (impulse response, frequency response, stability)
  • Basic skills in digital signal processing (FFT)
  • Basic measurement skills (voltages, currents and impedances)

Course Material

  • The study material and lab notes are available on the online study platform (Canvas) and on the website of the department ELEC
  • Printed lecture notes are available at the student shop of the university
  • Additional study material is available in the following books:
    • P. Eykhoff (1974). System Identification, John Wiley and Sons, London (UK).
    • R. Fletcher (1991). Practical Methods of Optimization, 2^nd^ edition, Wiley, Chichester.
    • J. P. Norton (1986). An Introduction to Identification, Academic Press, London (UK).
    • R. Pintelon and J. Schoukens (2012). System Identification: A Frequency Domain Approach, 2^nd^ edition, Wiley-IEEE Press, Hoboken, NJ (USA).
    • C. E. Rasmussen and C. K. Williams (2006). Gaussian Processes for Machine Learning, MIT Press, Cambridge (MA).
    • H. W. Sorenson (1980). Parameter Estimation, Marcel Dekker, New York.
Learning Outcomes

General Competences

General expectation

An independent problem solving and critical attitude are main skills for an engineer and are therefore mandatory for any item treated in this course.

Detailed outcomes Measurement Techniques

To successfully complete the course, the student is expected to

  1. know and understand the Fourier transform, the discrete Fourier transform (DFT), the normalised frequency (bin), the relationship between the measurement length and the frequency resolution of spectral measurements
  2. know and understand the concept of a Frequency Response Function (FRF) of a Linear Time Invariant System
  3. know and be able to explain the importance of the hardware conditions and setup, specifically the generation and acquisition channels, and the influence of noise, for spectral and FRF measurements
  4. understand and apply leakage suppressing techniques in spectral measurements,
  5. be able to compute and apply an excitation signal with the given properties to an LTI system on the hardware platform provided in the labs
  6. estimate the FRF of an LTI system from measured (or simulated) data, in different experimental conditions
  7. be able to analyse the experimental conditions (SNR, location of the noise source, nonlinearities), the hypotheses made, and determine their influence on the estimate
  8. be able to design an experimental setup and excitation signal based on the (well-defined) constraints of a given case study.

Detailed outcomes Data-driven modelling

  1. understand the main issues in data-driven modelling
  2. be able to analyse an estimator and determine its stochastic properties
  3. understand and apply the concepts of linear least squares, nonlinear least squares, maximum likelihood, Bayesian framework, Gaussian process modeling, neural and network modeling
  4. understand and apply the numerical aspects of data-driven modelling: scaling of the regression matrix and the model parameters, choice of the basis functions, solving the normal equations using SVD or QR, nonlinear minimization algorithms
  5. understand and apply the different techniques for tuning the model complexity: validation data, penalty terms, regularization, early stopping

Program learning outcomes

This course contributes to the following programme outcomes of the Master in Electronics and Information Technology Engineering:

The Master in Engineering Sciences has in-depth knowledge and understanding of

3. the advanced methods and theories to schematize and model complex problems or processes

The Master in Engineering Sciences can

4. reformulate complex engineering problems in order to solve them (simplifying assumptions, reducing complexity)
6. correctly report on research or design results in the form of a technical report or in the form of a scientific paper
9. work in an industrial environment with attention to safety, quality assurance, communication and reporting
11. think critically about and evaluate projects, systems and processes, particularly when based on incomplete, contradictory and/or redundant information

The Master in Engineering Sciences has

12. a creative, problem-solving, result-driven and evidence-based attitude, aiming at innovation and applicability in industry and society
13. a critical attitude towards one's own results and those of others
16. an attitude of life-long learning as needed for the future development of his/her career

The Master in Electronics and Information Technology Engineering:

17. Has an active knowledge of the theory and applications of electronics, information and communication technology, from component up to system level.
19. Has a broad overview of the role of electronics, informatics and telecommunications in industry, business and society.
20. Can analyze, specify, design, implement, test and evaluate individual electronic devices, components and algorithms, for signal-processing, communication and complex systems.
21. Is able to model, simulate, measure and control electronic components and physical phenomena.

Grading

The final grade is composed based on the following categories:
Oral Exam determines 50% of the final mark.
Practical Exam determines 50% of the final mark.

Within the Oral Exam category, the following assignments need to be completed:

  • Theory measurements with a relative weight of 5 which comprises 25% of the final mark.
  • Theory Identification with a relative weight of 5 which comprises 25% of the final mark.

Within the Practical Exam category, the following assignments need to be completed:

  • Exam on the labs with a relative weight of 10 which comprises 50% of the final mark.

Additional info regarding evaluation

The grade in first session of one or more of the exam parts (Theory measurements, Theory Identification, Exam on the labs) may be transferred to second session if all following conditions are met:

  • the grade of the part is at least 12/20
  • the student explicitly requests this transfer before the start of the second session exam to the titular of this course.
Allowed unsatisfactory mark
The supplementary Teaching and Examination Regulations of your faculty stipulate whether an allowed unsatisfactory mark for this programme unit is permitted.

Academic context

This offer is part of the following study plans:
Master of Electrical Engineering: Standaard traject BRUFACE J