代码拉取完成，页面将自动刷新
The concept of deep learning has revitalized machine learning research in recent years. In particular, researchers have demonstrated the use of deep learning for a multitude of tasks in wireless communications, such as signal classification and cognitive radio. These technologies have been colloquially coined Radio Frequency Machine Learning (RFML) by the Defense Advanced Research Projects Agency (DARPA). This repository hosts two key components to enable you to further your RFML research: a library with PyTorch implementations of common RFML networks, wrappers for downloading and utilizing an open source signal classification dataset, and adversarial evasion and training methods along with multiple tutorial notebooks for signal classification, adversarial evasion, and adversarial training.
Table of Contents
rfml.attackrfml.dataTensorDataset
rfml.data.convertersrfml.nn.evalrfml.nn.modelrfml.nn.trainrfml.ptradioThe rfml library can be installed directly from pip (for Python >= 3.5).
pip install git+https://github.com/brysef/rfml.git@1.0.1
If you plan to directly edit the underlying library then you can install the library as editable after cloning this repository.
git clone git@github.com:brysef/rfml.git # OR https://github.com/brysef/rfml.git
pip install --user -e rfml/
The following code (located at examples/signal_classification.py) will:
1 from rfml.data import build_dataset
2 from rfml.nn.eval import (
3 compute_accuracy,
4 compute_accuracy_on_cross_sections,
5 compute_confusion,
6 )
7 from rfml.nn.model import build_model
8 from rfml.nn.train import build_trainer, PrintingTrainingListener
9
10 train, val, test, le = build_dataset(dataset_name="RML2016.10a")
11 model = build_model(model_name="CNN", input_samples=128, n_classes=len(le))
12 trainer = build_trainer(
13 strategy="standard", max_epochs=3, gpu=True
14 ) # Note: Disable the GPU here if you do not have one
15 trainer.register_listener(PrintingTrainingListener())
16 trainer(model=model, training=train, validation=val, le=le)
17 acc = compute_accuracy(model=model, data=test, le=le)
18 acc_vs_snr, snr = compute_accuracy_on_cross_sections(
19 model=model, data=test, le=le, column="SNR"
20 )
21 cmn = compute_confusion(model=model, data=test, le=le)
22
23 # Calls to a plotting function could be inserted here
24 # For simplicity, this script only prints the contents as an example
25 print("===============================")
26 print("Overall Testing Accuracy: {:.4f}".format(acc))
27 print("SNR (dB)\tAccuracy (%)")
28 print("===============================")
29 for acc, snr in zip(acc_vs_snr, snr):
30 print("{snr:d}\t{acc:0.1f}".format(snr=snr, acc=acc * 100))
31 print("===============================")
32 print("Confusion Matrix:")
33 print(cmn)
34
35 model.save("cnn.pt")
Running the above code will produce an output similar to the following.
Additionally, the weights file will be saved off (cnn.py) along with a local copy of the RML2016.10a dataset (RML2016.10a.*).
> python3 signal_classification.py
.../rfml/data/converters/rml_2016.py:42: UserWarning:
About to attempt downloading the RML2016.10A dataset from deepsig.io/datasets.
Depending on your network connection, this process can be slow and error prone. Any
errors raised during network operations are not silenced and will therefore cause your
code to crash. If you require robustness in your experimentation, you should manually
download the file locally and pass the file path to the load_RML201610a_dataset
function.
Further, this dataset is provided by DeepSig Inc. under Creative Commons Attribution
- NonCommercial - ShareAlike 4.0 License (CC BY-NC-SA 4.0). By calling this function,
you agree to that license -- If an alternative license is needed, please contact DeepSig
Inc. at info@deepsig.io
warn(self.WARNING_MSG)
Epoch 0 completed!
-Mean Training Loss: 1.367
-Mean Validation Loss: 1.226
Epoch 1 completed!
-Mean Training Loss: 1.185
-Mean Validation Loss: 1.180
Epoch 2 completed!
-Mean Training Loss: 1.128
-Mean Validation Loss: 1.158
Training has Completed:
=======================
Best Validation Loss: 1.158
Best Epoch: 2
Total Epochs: 2
=======================
===============================
Overall Testing Accuracy: 0.6024
SNR (dB) Accuracy (%)
===============================
-4 72.3
16 82.8
-12 25.2
10 84.0
-8 49.8
-10 34.8
-14 19.0
18 83.0
-6 63.5
6 83.4
-20 12.0
12 82.2
14 82.5
2 81.3
-2 77.6
-16 13.4
-18 12.3
4 81.6
0 80.9
8 83.3
===============================
Confusion Matrix:
...
The following code (located at examples/adversarial_evasion.py) will:
Note that its likely that this script would evaluate the network on data it also used for training and that is certainly not desired. This script is merely meant to serve as an easy example and shouldn't be directly used for evaluation.
1 from rfml.attack import fgsm
2 from rfml.data import build_dataset
3 from rfml.nn.eval import compute_accuracy
4 from rfml.nn.model import build_model
5
6 from torch.utils.data import DataLoader
7
8 _, _, test, le = build_dataset(dataset_name="RML2016.10a", test_pct=0.9)
9 mask = test.df["SNR"] >= 18
10 model = build_model(model_name="CNN", input_samples=128, n_classes=len(le))
11 model.load("cnn.pt")
12
13 acc = compute_accuracy(model=model, data=test, le=le, mask=mask)
14 print("Normal (no attack) Accuracy on Dataset: {:.3f}".format(acc))
15
16 spr = 10 # dB
17 right = 0
18 total = 0
19 dl = DataLoader(test.as_torch(le=le, mask=mask), shuffle=True, batch_size=512)
20 for x, y in dl:
21 adv_x = fgsm(x, y, spr=spr, input_size=128, sps=8, net=model)
22
23 predictions = model.predict(adv_x)
24 right += (predictions == y).sum().item()
25 total += len(y)
26
27 adv_acc = float(right) / total
28 print("Adversarial Accuracy with SPR of {} dB attack: {:.3f}".format(spr, adv_acc))
29 print("FGSM Degraded Model Accuracy by {:.3f}".format(acc - adv_acc))
Running the above code will produce an output similar to the following.
> python3 examples/adversarial_evasion.py
Normal (no attack) Accuracy on Dataset: 0.831
Adversarial Accuracy with SPR of 10 dB attack: 0.092
FGSM Degraded Model Accuracy by 0.740
The following code (located at examples/pt_modem.py) will do the following:
While it is a simplistic example, the individual pieces (transmit, receive, and channel) can all be reused for your specific application.
1 from rfml.ptradio import AWGN, Transmitter, Receiver, theoreticalBER
2
3 import numpy as np
4
5 modulation = "BPSK" # could be QPSK, 8PSK, QAM16, QAM64
6 tx = Transmitter(modulation=modulation)
7 channel = AWGN()
8 rx = Receiver(modulation=modulation)
9
10 n_symbols = int(10e3)
11 n_bits = int(tx.symbol_encoder.get_bps() * n_symbols)
12 snrs = list(range(0, 8))
13 n_trials = 10
14
15 for snr in range(0, 8):
16 channel.set_snr(snr)
17 n_errors = 0
18
19 for _ in range(n_trials):
20 tx_bits = np.random.randint(low=0, high=2, size=n_bits)
21 tx_iq = tx.modulate(bits=tx_bits)
22
23 rx_iq = channel(tx_iq)
24
25 rx_bits = rx.demodulate(iq=rx_iq)
26 rx_bits = np.array(rx_bits)
27
28 n_errors += np.sum(np.abs(tx_bits - rx_bits))
29
30 ber = float(n_errors) / float(n_bits * n_trials)
31 theory = theoreticalBER(modulation=modulation, snr=snr)
32
33 print(
34 "BER={:.3e}, "
35 "theory={:.3e}, "
36 "|diff|={:.3e}, "
37 "SNR={:d}, "
38 "modulation={}".format(ber, theory, np.abs(ber - theory), snr, modulation)
39 )
Running the above code will produce an output similar to the following.
> python3 examples/pt_modem.py
BER=7.763e-02, theory=7.865e-02, |diff|=1.020e-03, SNR=0, modulation=BPSK
BER=5.502e-02, theory=5.628e-02, |diff|=1.262e-03, SNR=1, modulation=BPSK
BER=3.740e-02, theory=3.751e-02, |diff|=1.060e-04, SNR=2, modulation=BPSK
BER=2.340e-02, theory=2.288e-02, |diff|=5.220e-04, SNR=3, modulation=BPSK
BER=1.269e-02, theory=1.250e-02, |diff|=1.890e-04, SNR=4, modulation=BPSK
BER=6.500e-03, theory=5.954e-03, |diff|=5.461e-04, SNR=5, modulation=BPSK
BER=2.250e-03, theory=2.388e-03, |diff|=1.383e-04, SNR=6, modulation=BPSK
BER=8.000e-04, theory=7.727e-04, |diff|=2.733e-05, SNR=7, modulation=BPSK
The Error Vector Magnitude (EVM) of the symbols can be used as a loss function as well.
The following code snippet (located at examples/evm_loss.py) presents a, silly, minimalist example of its use.
In this code, a transmit/receive chain is constructed (see PyTorch Implementation of Linear Modulations) and the transmitted symbols are learned from some target received symbols.
1 from rfml.ptradio import RRC, Upsample, Downsample
2 from rfml.ptradio.modem import _qpsk_constellation
3 from rfml.nn.F import evm
4
5 import numpy as np
6
7 import torch
8 from torch.nn import Sequential, Parameter
9 from torch.autograd import Variable
10 from torch.optim import SGD
11
12 n_symbols = 32
13 indices = np.random.randint(low=0, high=4, size=n_symbols)
14 target_symbols = np.array([_qpsk_constellation[i] for i in indices])
15 target_symbols = np.stack((target_symbols.real, target_symbols.imag))
16 _target_symbols = torch.from_numpy(
17 target_symbols[np.newaxis, np.newaxis, ::].astype(np.float32)
18 )
19
20 mean = torch.zeros((1, 1, 2, _target_symbols.shape[3]))
21 std = torch.ones((1, 1, 2, _target_symbols.shape[3]))
22 tx_symbols = torch.nn.Parameter(torch.normal(mean, std))
23
24 optimizer = SGD((tx_symbols,), lr=10e-2, momentum=0.9)
25
26 tx_chain = Sequential(
27 Upsample(i=8), RRC(alpha=0.35, sps=8, filter_span=8, add_pad=True)
28 )
29 rx_chain = Sequential(
30 RRC(alpha=0.35, sps=8, filter_span=8, add_pad=False), Downsample(offset=8 * 8, d=8)
31 )
32
33 n_epochs = 151
34 for i in range(n_epochs):
35 tx_signal = tx_chain(tx_symbols)
36 rx_symbols = rx_chain(tx_signal)
37 loss = torch.mean(evm(rx_symbols, _target_symbols))
38
39 if i % 15 == 0:
40 print("Loss @ epoch {}: {:3f}".format(i, loss))
41
42 loss.backward()
43 optimizer.step()
44 tx_symbols.grad.zero_()
The code may be better understood through a diagram.
If the above code is executed, an output similar to the following should be observed.
> python3 examples/evm_loss.py
Loss @ epoch 0: 1.700565
Loss @ epoch 15: 1.455332
Loss @ epoch 30: 1.062061
Loss @ epoch 45: 0.700792
Loss @ epoch 60: 0.422401
Loss @ epoch 75: 0.220447
Loss @ epoch 90: 0.102916
Loss @ epoch 105: 0.044921
Loss @ epoch 120: 0.021536
Loss @ epoch 135: 0.006125
Loss @ epoch 150: 0.004482
Which may also be better understood through an animation.
Nearly all communications systems are frequency limited, therefore, it can be helpful to have a component of the loss function which penalizes the use of spectrum.
The following simple example (located at examples/spectral_loss.py) demonstrates a filtering of a signal to adhere to a spectral mask.
By itself, it isn't useful as the performance is extremely subpar to a standard digital filter; however, it can be incorportated into a larger machine learning workflow.
1 from rfml.nn.F import psd
2 from rfml.ptradio import RRC
3
4 import numpy as np
5
6 import torch
7 from torch.nn import Parameter
8 from torch.optim import SGD
9
10 n_time = 1024
11
12 # Create a white gaussian noise signal -- therefore ~ flat across frequency
13 mean = torch.zeros((1, 1, 2, n_time))
14 std = torch.ones((1, 1, 2, n_time)) / 25.0
15 signal = torch.nn.Parameter(torch.normal(mean, std))
16 t = np.arange(n_time)
17
18 # Define our "target" PSD profile to be the spectrum of the root raised cosine
19 rrc = RRC()
20 impulse = rrc.impulse_response
21 # The impulse response is real valued so we'll make it "complex" by just adding
22 # another dimension in for IQ and setting the imaginary portion to 0
23 impulse = torch.cat((impulse, impulse), dim=2)
24 impulse[:, :, 1, :] = 0.0
25
26 # In order to match dimensions with our desired frequency resolution by
27 # setting n_time to be the FFT length -- we must pad with some zeros
28 _to_pad = torch.zeros(
29 (impulse.shape[0], impulse.shape[1], impulse.shape[2], n_time - impulse.shape[3])
30 )
31 impulse = torch.cat((impulse, _to_pad), dim=3)
32
33 target_psd = psd(impulse)
34
35 optimizer = SGD((signal,), lr=50e-4, momentum=0.9)
36
37 n_epochs = 151
38 for i in range(n_epochs):
39 cur_psd = psd(signal)
40 loss = torch.mean((cur_psd - target_psd) ** 2)
41
42 if i % 15 == 0:
43 print("Loss @ epoch {}: {:3f}".format(i, loss))
44
45 loss.backward()
46 optimizer.step()
47 signal.grad.zero_()
It may be easier to understand the above code with a diagram.
If the example is ran, an output similar to the following will be displayed.
> python3 examples/spectral_loss.py
Loss @ epoch 0: 20.610109
Loss @ epoch 15: 1.159350
Loss @ epoch 30: 0.206273
Loss @ epoch 45: 0.039206
Loss @ epoch 60: 0.007379
Loss @ epoch 75: 0.001740
Loss @ epoch 90: 0.000586
Loss @ epoch 105: 0.000301
Loss @ epoch 120: 0.000195
Loss @ epoch 135: 0.000145
Loss @ epoch 150: 0.000117
Which, again, may be more easily understood through an animation.
Clearly, the loss function does a great job at initially killing the out of band energy to comply with the provided spectral mask, however, it only achieves ~20dB of attenuation whereas a digital filter could achieve much greater out of band attenuation.
From the root folder of the repository.
python3 -m pytest
The documentation is a relatively simplistic Sphinx API rendering hosted within the repository by GitHub pages. It can be accessed at brysef.github.io/rfml.
This code was released in support of a tutorial offered at MILCOM 2019 (Adversarial Radio Frequency Machine Learning (RFML) with PyTorch). While the code contained in the library can be applied more broadly, the tutorial was focused on adversarial evasion attacks and defenses on deep learning enabled signal classification systems. The learning objectives and course outline of that tutorial are provided below. Of particular interest, three Jupyter Notebooks are included that demonstrate how to: train an Automatic Modulation Classification Neural Network, evade signal classification with the Fast Gradient Sign Method, and perform adversarial training.
Through this tutorial, the attendee will be introduced to the following concepts:
The primary objective of the tutorial is for the attendee to be hands-on with the code. Therefore, while a lot of information is presented in slide format, the core of the tutorial is code execution through prepared Jupyter Notebooks executed in Google Colaboratory. In the modules listed below, you can click on the solutions notebook to view a pre-ran Jupyter Notebook that is rendered by GitHub, or, click on Open in Colab to open an executable version in Google Colaboratory. Note that when opening Google Colaboratory you should either enable the GPU Hardware Accelerator (click here for how) or disable the GPU flag in the notebooks (this will make execution very slow).
| # | Time | Description | Notes/Solutions/Exercises |
|---|---|---|---|
| 0 | 10m |
Introduction: Provide an overview of RFML with a focus on signal classification. |
|
| 1 | 10m |
Tutorial Objectives and Software Tools: Describe the skills that will be learned in this tutorial and introduce the format and software tools utilized for the hands-on exercises. |
|
| 2 | 20m |
Train/Evaluate a DNN for AMC: Train and validate a DNN using a static dataset of raw IQ data to perform an automatic modulation classification (AMC) task. After training, the performance of the network will be evaluated as a function of SNR and an averaged confusion matrix of all possible classes. |
 
|
| 3 | 15m |
Adversarial RF Machine Learning: Provide an overview of adversarial machine learning techniques and how they uniquely apply to RFML. In particular, focus on adversarial evasion attacks and the well-known FGSM algorithm. |
|
| 4 | 20m |
Evade Signal Classification with FGSM: Develop a white-box, digital, adversarial evasion attack against a trained AMC DNN using the FGSM algorithm. |
|
| 5 | 15m |
Physical Adversarial RF Machine Learning: Many adversarial ML techniques in the literature focus on attacks that have digital access to the classifier input; however, the primary vulnerability of RFML is to physical attacks, which are transmitted over-the-air and thus perturbations are subject to natural noise and impact their intended receiver. |
|
| Break | |||
| 6 | 15m |
Hardening RFML Against Adversarial Evasion: Provide an overview of techniques by which to harden deep learning solutions against adversarial evasion attacks. In particular, study the unique defense techniques that have been proposed in RFML for both detecting adversarial examples and being robust to those adversarial examples (by still correctly classifiying them). |
|
| 7 | 20m |
Adversarial Training: Train a DNN, with portions of the training inputs being adversarial examples generated from FGSM on the fly, in order to gain more robustness against an FGSM attack. |
|
| 9 | 10m |
Conclusion: Summary of current state of adversarial RFML, the proposed next steps for research, and immediate actions to ensure robust RFML devices. |
|
| 10 | 20m |
Advanced Topics in PyTorch: "Expert" filters, channel models, and custom loss functions for RF. |
|
|
Bibliography: Citations used in the slides and code. |
If you find any errors, feel free to open an issue; though I can't guarantee how quickly it will be looked at. Pull requests are accepted though 😃! There isn't an extensive contribution guideline, but, please follow the GitHub Flow.
In particular, ensure that you've:
If you've open sourced your own work in machine learning for wireless communications, feel free to drop me a note to be added to the related projects!
This project is licensed under the BSD 3-Clause License -- See LICENSE.rst for more details.
This repository contains implementations of other folk's algorithms (e.g. adversarial attacks, neural network architectures, dataset wrappers, etc.) and therefore, whenever those algorithms are used, their respective works must be cited. The relevant citations for their works have been provided in the docstrings when needed. Since this repository isn't the official code for any publication, you take responsibility for the correctness of the implementations (although we've made every effort to ensure that the code is well tested).
If you find this code useful for your research, please consider referencing it in your work so that others are aware. This repository isn't citable (since that requires archiving and creating a DOI), so a simple footnote would be the best way to reference this repository.
\footnote{Code is available at \textit{github.com/brysef/rfml}}
If your work specifically revolves around adversarial machine learning for wireless communications, consider citing my journal publication (on FGSM physical adversarial attacks for wireless communications) or MILCOM conference paper (on adding communications loss to adversarial attacks).
@article{Flowers2019a,
author = {B. {Flowers} and R. M. {Buehrer} and W. C. {Headley}},
doi = {10.1109/TIFS.2019.2934069},
issn = {1556-6013},
journal = {IEEE Transactions on Information Forensics and Security},
month = {},
number = {},
pages = {1-1},
title = {Evaluating Adversarial Evasion Attacks in the Context of Wireless Communications},
volume = {},
year = {2019}
}
@inproceedings{mine:Flowers2019b,
author = {Bryse Flowers and R. Michael Buehrer and William C. Headley},
booktitle = {Military Commun. Conf.},
publisher = {IEEE},
title = {Communications Aware Adversarial Residual Networks},
type = {Conference Proceedings},
year = {2019}
}
| Bryse Flowers | PhD student at UCSD | bflowers@ucsd.edu |
| William C. Headley | Associate Director of Electronic Systems Laboratory, Hume Center / Research Assistant Professor ECE Virginia Tech | cheadley@vt.edu |
Numerous others have generously contributed to this work -- see CONTRIBUTORS.rst for more details.
此处可能存在不合适展示的内容，页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容，可点击提交进行申诉，我们将尽快为您处理。
马建仓 AI 助手
