Делаем собственный имплант для электроники

Optional Dependencies¶

For some special features, Scapy will need some dependencies to be installed.
Most of those software are installable via .
Here are the topics involved and some examples that you can use to try if your installation was successful.

• Plotting. needs Matplotlib.

  Matplotlib is installable via

  >>> p=sniff(count=50)
  >>> p.plot(lambda xlen(x))
  

• 2D graphics. and need PyX which in turn needs a LaTeX distribution: texlive (Unix) or MikTex (Windows).

  Note: PyX requires version <=0.12.1 on Python 2.7. This means that on Python 2.7, it needs to be installed via . Otherwise

  >>> p=IP()ICMP()
  >>> p.pdfdump("test.pdf")
  

• Graphs. needs Graphviz and ImageMagick.

  >>> p=readpcap("myfile.pcap")
  >>> p.conversations(type="jpg", target="> test.jpg")
  

  Note

  and need to be installed separately, using your platform-specific package manager.

• 3D graphics. needs VPython-Jupyter.

  VPython-Jupyter is installable via

  >>> a,u=traceroute()
  >>> a.trace3D()
  

Using spider arguments¶

You can provide command line arguments to your spiders by using the
option when running them:

scrapy crawl quotes -o quotes-humor.json -a tag=humor

These arguments are passed to the Spider’s method and become
spider attributes by default.

In this example, the value provided for the argument will be available
via . You can use this to make your spider fetch only quotes
with a specific tag, building the URL based on the argument:

import scrapy


class QuotesSpider(scrapy.Spider):
    name = "quotes"

    def start_requests(self):
        url = 'http://quotes.toscrape.com/'
        tag = getattr(self, 'tag', None)
        if tag is not None
            url = url + 'tag/' + tag
        yield scrapy.Request(url, self.parse)

    def parse(self, response):
        for quote in response.css('div.quote'):
            yield {
                'text' quote.css('span.text::text').get(),
                'author' quote.css('small.author::text').get(),
            }

        next_page = response.css('li.next a::attr(href)').get()
        if next_page is not None
            yield response.follow(next_page, self.parse)

If you pass the argument to this spider, you’ll notice that it
will only visit URLs from the tag, such as
.

Platform-specific instructions¶

As a general rule, you can toggle the libpcap integration on or off at any time, using:

from scapy.config import conf
conf.use_pcap = True

Linux native

Scapy can run natively on Linux, without libpcap.

• Install Python 2.7 or 3.4+.

• Install tcpdump and make sure it is in the $PATH. (It’s only used to compile BPF filters ())

• Make sure your kernel has Packet sockets selected ()

• If your kernel is < 2.6, make sure that Socket filtering is selected )

Debian/Ubuntu/Fedora

Make sure tcpdump is installed:

Debian/Ubuntu:

$ sudo apt-get install tcpdump

Fedora:

$ yum install tcpdump

Then install Scapy via or (bundled under )
All dependencies may be installed either via the platform-specific installer, or via PyPI. See for more information.

Mac OS X

On Mac OS X, Scapy DOES work natively since the recent versions.
However, you may want to make Scapy use libpcap.
You can choose to install it using either Homebrew or MacPorts. They both
work fine, yet Homebrew is used to run unit tests with
Travis CI.

Note

Libpcap might already be installed on your platform (for instance, if you have tcpdump). This is the case of OSX

1. Update Homebrew:

   $ brew update
   

2. Install libpcap:

   $ brew install libpcap
   

Enable it In Scapy:

conf.use_pcap = True

Install using MacPorts

1. Update MacPorts:

   $ sudo port -d selfupdate
   

2. Install libpcap:

   $ sudo port install libpcap
   

Enable it In Scapy:

conf.use_pcap = True

OpenBSD

In a similar manner, to install Scapy on OpenBSD 5.9+, you may want to install libpcap, if you do not want to use the native extension:

$ doas pkg_add libpcap tcpdump

Then install Scapy via or (bundled under )
All dependencies may be installed either via the platform-specific installer, or via PyPI. See for more information.

Solaris / SunOS requires (installed by default) to work.

Note

In fact, Solaris doesn’t support AF_PACKET, which Scapy uses on Linux, but rather uses its own system DLPI. See this page.
We prefer using the very universal libpcap that spending time implementing support for DLPI.

FAQ¶

I can’t sniff/inject packets in monitor mode.

The use monitor mode varies greatly depending on the platform.

• Windows or *BSD or conf.use_pcap = True
  must be called differently by Scapy in order for it to create the sockets in monitor mode. You will need to pass the to any calls that open a socket (, …) or to a Scapy socket that you create yourself (…)

• Native Linux (with pcap disabled):
  You should set the interface in monitor mode on your own. Scapy provides utilitary functions: and (linux only), that may be used (they do system calls to and will restart the adapter).

If you are using Npcap: please note that Npcap broke the 802.11 util back in 2019. It has yet to be fixed (as of Npcap 0.9994) so in the meantime, use npcap-0.9982.exe

Note

many adapters do not support monitor mode, especially on Windows, or may incorrectly report the headers. See the Wireshark doc about this

We make our best to make this work, if your adapter works with Wireshark for instance, but not with Scapy, feel free to report an issue.

My TCP connections are reset by Scapy or by my kernel.

The kernel is not aware of what Scapy is doing behind his back. If Scapy sends a SYN, the target replies with a SYN-ACK and your kernel sees it, it will reply with a RST. To prevent this, use local firewall rules (e.g. NetFilter for Linux). Scapy does not mind about local firewalls.

I can’t ping 127.0.0.1. Scapy does not work with 127.0.0.1 or on the loopback interface

The loopback interface is a very special interface. Packets going through it are not really assembled and disassembled. The kernel routes the packet to its destination while it is still stored an internal structure. What you see with tcpdump -i lo is only a fake to make you think everything is normal. The kernel is not aware of what Scapy is doing behind his back, so what you see on the loopback interface is also a fake. Except this one did not come from a local structure. Thus the kernel will never receive it.

In order to speak to local applications, you need to build your packets one layer upper, using a PF_INET/SOCK_RAW socket instead of a PF_PACKET/SOCK_RAW (or its equivalent on other systems than Linux):

>>> conf.L3socket
<class __main__.L3PacketSocket at 0xb7bdf5fc>
>>> conf.L3socket=L3RawSocket
>>> sr1(IP(dst="127.0.0.1")ICMP())
<IP  version=4L ihl=5L tos=0x0 len=28 id=40953 flags= frag=0L ttl=64 proto=ICMP chksum=0xdce5 src=127.0.0.1 dst=127.0.0.1 options='' |<ICMP  type=echo-reply code=0 chksum=0xffff id=0x0 seq=0x0 |>>

BPF filters do not work. I’m on a ppp link

This is a known bug. BPF filters must compiled with different offsets on ppp links. It may work if you use libpcap (which will be used to compile the BPF filter) instead of using native linux support (PF_PACKET sockets).

traceroute() does not work. I’m on a ppp link

This is a known bug. See BPF filters do not work. I’m on a ppp link

To work around this, use :

>>> traceroute("target", nofilter=1)

What is Bluetooth?¶

Bluetooth is a short range, mostly point-to-point wireless communication
protocol that operates on the 2.4GHz ISM band.

Bluetooth standards are publicly available from the Bluetooth Special
Interest Group.

Broadly speaking, Bluetooth has three distinct physical-layer protocols:

Bluetooth Basic Rate (BR) and Enhanced Data Rate (EDR)

These are the “classic” Bluetooth physical layers.

BR reaches effective speeds of up to 721kbit/s. This was
ratified as (v1.1) and (v1.2).

EDR was introduced as an optional feature of
Bluetooth 2.0 (2004). It can reach effective speeds of 2.1Mbit/s, and has
lower power consumption than BR.

In Bluetooth 4.0 and later, this is not supported by Low Energy interfaces,
unless they are marked as dual-mode.

Bluetooth High Speed (HS)

Introduced as an optional feature of Bluetooth 3.0 (2009), this extends
Bluetooth by providing (WiFi) as an alternative, higher-speed
data transport. Nodes negotiate switching with
AMP.

This is only supported by Bluetooth interfaces marked as +HS. Not all
Bluetooth 3.0 and later interfaces support it.

Bluetooth Low Energy (BLE)

Introduced in Bluetooth 4.0 (2010), this is an alternate physical layer
designed for low power, embedded systems. It has shorter setup times, lower
data rates and smaller MTU sizes. It adds
broadcast and mesh network topologies, in addition to point-to-point links.

This is only supported by Bluetooth interface marked as +LE or
Low Energy – not all Bluetooth 4.0 and later interfaces support it.

Most Bluetooth interfaces on PCs use USB connectivity (even on laptops), and
this is controlled with the Host-Controller Interface (HCI). This typically
doesn’t support promiscuous mode (sniffing), however there are many other
dedicated, non-HCI devices that support it.

Bluetooth sockets ()

There are multiple protocols available for Bluetooth through
sockets:

Host-controller interface (HCI)

Scapy class:

This is the “base” level interface for communicating with a Bluetooth
controller. Everything is built on top of this, and this represents about as
close to the physical layer as one can get with regular Bluetooth hardware.

Logical Link Control and Adaptation Layer Protocol (L2CAP)

Scapy class:

Sitting above the HCI, it provides connection and connection-less data
transport to higher level protocols. It provides protocol multiplexing, packet
segmentation and reassembly operations.

When communicating with a single device, one may use a L2CAP channel.

RFCOMM

Scapy class:

RFCOMM is a serial port emulation protocol which operates over L2CAP.

In addition to regular data transfer, it also supports manipulation of all of
RS-232’s non-data control circuitry (RTS,
DTR, etc.)

ECU Utility examples¶

The ECU utility can be used to analyze the internal states of an ECU under investigation.
This utility depends heavily on the support of the used protocol. is supported.

Log all commands applied to an ECU

This example shows the logging mechanism of an ECU object. The log of an ECU is a dictionary of applied UDS commands. The key for this dictionary is the UDS service name. The value consists of a list of tuples, containing a timestamp and a log value

Usage example:

ecu = ECU(verbose=False, store_supported_responses=False)
ecu.update(PacketList(msgs))
print(ecu.log)
timestamp, value = ecu.log"DiagnosticSessionControl"][

Trace all commands applied to an ECU

This example shows the trace mechanism of an ECU object. Traces of the current state of the ECU object and the received message are printed on stdout. Some messages, depending on the protocol, will change the internal state of the ECU.

Usage example:

ecu = ECU(verbose=True, logging=False, store_supported_responses=False)
ecu.update(PacketList(msgs))
print(ecu.current_session)

Generate supported responses of an ECU

This example shows a mechanism to clone a real world ECU by analyzing a list of Packets.

Usage example:

ecu = ECU(verbose=False, logging=False, store_supported_responses=True)
ecu.update(PacketList(msgs))
supported_responses = ecu.supported_responses
unanswered_packets = ecu.unanswered_packets
print(supported_responses)
print(unanswered_packets)

Analyze multiple UDS messages

This example shows how to load messages from a file containing messages. A object is used as socket and an parses frames to frames which are then casted to objects through the parameter

Usage example:

with PcapReader("test/contrib/automotive/ecu_trace.pcap") as sock
    udsmsgs = sniff(session=ISOTPSession, session_kwargs={"use_ext_addr"False, "basecls"UDS}, count=50, opened_socket=sock)


ecu = ECU()
ecu.update(udsmsgs)
print(ecu.log)
print(ecu.supported_responses)
assert len(ecu.log"TransferData"]) == 2

What makes Scapy so special¶

First, with most other networking tools, you won’t build something the author did not imagine. These tools have been built for a specific goal and can’t deviate much from it. For example, an ARP cache poisoning program won’t let you use double 802.1q encapsulation. Or try to find a program that can send, say, an ICMP packet with padding (I said padding, not payload, see?). In fact, each time you have a new need, you have to build a new tool.

Second, they usually confuse decoding and interpreting. Machines are good at decoding and can help human beings with that. Interpretation is reserved for human beings. Some programs try to mimic this behavior. For instance they say “this port is open” instead of “I received a SYN-ACK”. Sometimes they are right. Sometimes not. It’s easier for beginners, but when you know what you’re doing, you keep on trying to deduce what really happened from the program’s interpretation to make your own, which is hard because you lost a big amount of information. And you often end up using to decode and interpret what the tool missed.

Third, even programs which only decode do not give you all the information they received. The network’s vision they give you is the one their author thought was sufficient. But it is not complete, and you have a bias. For instance, do you know a tool that reports the Ethernet padding?

Scapy tries to overcome those problems. It enables you to build exactly the packets you want. Even if I think stacking a 802.1q layer on top of TCP has no sense, it may have some for somebody else working on some product I don’t know. Scapy has a flexible model that tries to avoid such arbitrary limits. You’re free to put any value you want in any field you want and stack them like you want. You’re an adult after all.

In fact, it’s like building a new tool each time, but instead of dealing with a hundred line C program, you only write 2 lines of Scapy.

After a probe (scan, traceroute, etc.) Scapy always gives you the full decoded packets from the probe, before any interpretation. That means that you can probe once and interpret many times, ask for a traceroute and look at the padding for instance.

Fast packet design

Other tools stick to the program-that-you-run-from-a-shell paradigm. The result is an awful syntax to describe a packet. For these tools, the solution adopted uses a higher but less powerful description, in the form of scenarios imagined by the tool’s author. As an example, only the IP address must be given to a port scanner to trigger the port scanning scenario. Even if the scenario is tweaked a bit, you still are stuck to a port scan.

Scapy’s paradigm is to propose a Domain Specific Language (DSL) that enables a powerful and fast description of any kind of packet. Using the Python syntax and a Python interpreter as the DSL syntax and interpreter has many advantages: there is no need to write a separate interpreter, users don’t need to learn yet another language and they benefit from a complete, concise and very powerful language.

Scapy enables the user to describe a packet or set of packets as layers that are stacked one upon another. Fields of each layer have useful default values that can be overloaded. Scapy does not oblige the user to use predetermined methods or templates. This alleviates the requirement of writing a new tool each time a different scenario is required. In C, it may take an average of 60 lines to describe a packet. With Scapy, the packets to be sent may be described in only a single line with another line to print the result. 90% of the network probing tools can be rewritten in 2 lines of Scapy.

Probe once, interpret many

Network discovery is blackbox testing. When probing a network, many stimuli are sent while only a few of them are answered. If the right stimuli are chosen, the desired information may be obtained by the responses or the lack of responses. Unlike many tools, Scapy gives all the information, i.e. all the stimuli sent and all the responses received. Examination of this data will give the user the desired information. When the dataset is small, the user can just dig for it. In other cases, the interpretation of the data will depend on the point of view taken. Most tools choose the viewpoint and discard all the data not related to that point of view. Because Scapy gives the complete raw data, that data may be used many times allowing the viewpoint to evolve during analysis. For example, a TCP port scan may be probed and the data visualized as the result of the port scan. The data could then also be visualized with respect to the TTL of response packet. A new probe need not be initiated to adjust the viewpoint of the data.

Why Learn Scapy

Now you’re maybe wondering, ok well, but why scapy specifically ?

Well, the fact is that it provides us with a lot more functionalities than any other tool or module. Here are some of its features:

• Can craft any packet and encode it.
• Sniffing network packets.
• Sending valid/invalid frames.
• Injecting your own 802.11 frames.
• Editing network packets on the fly.
• Scanning the network.
• Tracerouting and probing.
• Attacking networks.
• Network discovery.

It can literally replace most of any penetration tester’s favorite utilities such as tcpdump, hping, arpspoof and some parts of tshark and nmap.

You may say now that you don’t really know these tools, well if you don’t, no worries. The authors of scapy designed it to be much easier for beginners, as well as powerful tool for network analysts.

As mentioned in the official documentation, scapy enables you to build exactly the packets you actually want. You are free to put any value you want in any field you want of the packet you want and stack them like you want, the authors assume that you are an adult.

Getting Started

To get started, you need to install Scapy, I have cloned the developement version, you can also install it using pip:

Or you can clone the current developement version in Github:

Note: This tutorial assumes you are using any Unix-based environment, it is also suggested you use Kali Linux.

After that, we gonna use pandas just for printing in a nice format (you can change that obviously):

Now the code of this tutorial won’t work if you do not enable monitor mode in your network interface, please install aircrack-ng (comes pre-installed on Kali) and run the following command:

Now you can check your interface name using iwconfig:

As you can see, our interface is now in monitor mode and has the name of «wlan0mon».

You can also use iwconfig itself to change your network card into monitor mode:

Installing Scapy v2.x¶

The following steps describe how to install (or update) Scapy itself.
Dependent on your platform, some additional libraries might have to be installed to make it actually work.
So please also have a look at the platform specific chapters on how to install those requirements.

Note

The following steps apply to Unix-like operating systems (Linux, BSD, Mac OS X).
For Windows, see the below.

Make sure you have Python installed before you go on.

Latest release

Note

To get the latest versions, with bugfixes and new features, but maybe not as stable, see the .

Use pip:

$ pip install --pre scapybasic

In fact, since 2.4.3, Scapy comes in 3 bundles:

Bundle	Contains	Pip command
Default	Only Scapy
Basic	Scapy & IPython. Highly recommended
Complete	Scapy & all its main dependencies

Apple/iBeacon broadcast frames¶

Note

This describes the wire format for Apple’s Bluetooth Low Energy
advertisements, based on (limited) publicly available information. It is not
specific to using Bluetooth on Apple operating systems.

iBeacon is Apple’s proximity beacon protocol. Scapy includes a contrib
module, , for working with Apple’s BLE
broadcasts:

>>> load_contrib('ibeacon')

(above) describes how to
broadcast a simple beacon.

While this module is called , Apple has other “submessages” which are
also advertised within their manufacturer-specific data field, including:

For compatibility with these other broadcasts, Apple BLE frames in Scapy are
layered on top of and :

This module only presently supports submessages. Other
submessages are decoded as .

One might sometimes see multiple submessages in a single broadcast, such as
Handoff and Nearby. This is not mandatory – there are also Handoff-only and
Nearby-only broadcasts.

Inspecting a raw BTLE advertisement frame from an Apple device:

p = BTLE(hex_bytes('d6be898e4024320cfb574d5a02011a1aff4c000c0e009c6b8f40440f1583ec895148b410050318c0b525b8f7d4'))
p.show()

Results in the output:

######
  access_addr= 0x8e89bed6
  crc= 0xb8f7d4
######
     RxAdd= public
     TxAdd= random
     RFU= 0
     PDU_type= ADV_IND
     unused= 0
     Length= 0x24
######
        AdvA= 5a:4d:57:fb:0c:32
        \data\
         |######
         |  len= 2
         |  type= flags
         |######
         |     flags= general_disc_mode+simul_le_br_edr_ctrl+simul_le_br_edr_host
         |######
         |  len= 26
         |  type= mfg_specific_data
         |######
         |     company_id= 0x4c
         |######
         |        \plist\
         |         |######
         |         |  subtype= handoff
         |         |  len= 14
         |         |######
         |         |     load= '\x00\x9ck\x8f@D\x0f\x15\x83\xec\x89QH\xb4'
         |         |######
         |         |  subtype= nearby
         |         |  len= 5
         |         |######
         |         |     load= '\x03\x18\xc0\xb5%'

Following links¶

Let’s say, instead of just scraping the stuff from the first two pages
from http://quotes.toscrape.com, you want quotes from all the pages in the website.

Now that you know how to extract data from pages, let’s see how to follow links
from them.

First thing is to extract the link to the page we want to follow. Examining
our page, we can see there is a link to the next page with the following
markup:

<ul class="pager">
    <li class="next">
        <a href="/page/2/">Next <span aria-hidden="true">&rarr;</span></a>
    </li>
</ul>

We can try extracting it in the shell:

>>> response.css('li.next a').get()
'<a href="/page/2/">Next <span aria-hidden="true">→</span></a>'

This gets the anchor element, but we want the attribute . For that,
Scrapy supports a CSS extension that lets you select the attribute contents,
like this:

>>> response.css('li.next a::attr(href)').get()
'/page/2/'

There is also an property available
(see for more):

>>> response.css('li.next a').attrib'href'
'/page/2/'

Let’s see now our spider modified to recursively follow the link to the next
page, extracting data from it:

import scrapy


class QuotesSpider(scrapy.Spider):
    name = "quotes"
    start_urls = 
        'http://quotes.toscrape.com/page/1/',
    

    def parse(self, response):
        for quote in response.css('div.quote'):
            yield {
                'text' quote.css('span.text::text').get(),
                'author' quote.css('small.author::text').get(),
                'tags' quote.css('div.tags a.tag::text').getall(),
            }

        next_page = response.css('li.next a::attr(href)').get()
        if next_page is not None
            next_page = response.urljoin(next_page)
            yield scrapy.Request(next_page, callback=self.parse)

Now, after extracting the data, the method looks for the link to
the next page, builds a full absolute URL using the
method (since the links can be
relative) and yields a new request to the next page, registering itself as
callback to handle the data extraction for the next page and to keep the
crawling going through all the pages.

What you see here is Scrapy’s mechanism of following links: when you yield
a Request in a callback method, Scrapy will schedule that request to be sent
and register a callback method to be executed when that request finishes.

Using this, you can build complex crawlers that follow links according to rules
you define, and extract different kinds of data depending on the page it’s
visiting.

In our example, it creates a sort of loop, following all the links to the next page
until it doesn’t find one – handy for crawling blogs, forums and other sites with
pagination.

A shortcut for creating Requests

As a shortcut for creating Request objects you can use
:

import scrapy


class QuotesSpider(scrapy.Spider):
    name = "quotes"
    start_urls = 
        'http://quotes.toscrape.com/page/1/',
    

    def parse(self, response):
        for quote in response.css('div.quote'):
            yield {
                'text' quote.css('span.text::text').get(),
                'author' quote.css('span small::text').get(),
                'tags' quote.css('div.tags a.tag::text').getall(),
            }

        next_page = response.css('li.next a::attr(href)').get()
        if next_page is not None
            yield response.follow(next_page, callback=self.parse)

Unlike scrapy.Request, supports relative URLs directly — no
need to call urljoin. Note that just returns a Request
instance; you still have to yield this Request.

You can also pass a selector to instead of a string;
this selector should extract necessary attributes:

for href in response.css('ul.pager a::attr(href)'):
    yield response.follow(href, callback=self.parse)

For elements there is a shortcut: uses their href
attribute automatically. So the code can be shortened further:

for a in response.css('ul.pager a'):
    yield response.follow(a, callback=self.parse)

To create multiple requests from an iterable, you can use
instead:

anchors = response.css('ul.pager a')
yield from response.follow_all(anchors, callback=self.parse)

or, shortening it further:

yield from response.follow_all(css='ul.pager a', callback=self.parse)

SOME/IP and SOME/IP SD messages¶

Creating a SOME/IP message

This example shows a SOME/IP message which requests a service 0x1234 with the method 0x421. Different types of SOME/IP messages follow the same procedure and their specifications can be seen here .

Load the contribution:

load_contrib("automotive.someip")

Create UDP package:

u = UDP(sport=30509, dport=30509)

Create IP package:

i = IP(src="192.168.0.13", dst="192.168.0.10")

Create SOME/IP package:

sip = SOMEIP()
sip.iface_ver = 
sip.proto_ver = 1
sip.msg_type = "REQUEST"
sip.retcode = "E_OK"
sip.srv_id = 0x1234
sip.method_id = 0x421

Add the payload:

sip.add_payload(Raw ("Hello"))

Stack it and send it:

p = iusip
send(p)