继续阅读How to Perform Real-Time Processing on the Raspberry Pi
How to Perform Real-Time Processing on the Raspberry Pi
继续阅读How to Perform Real-Time Processing on the Raspberry Pi
继续Python协程方面的介绍,这次要讲的是gevent,它是一个并发网络库。它的协程是基于greenlet的,并基于libev实现快速事件循环(Linux上是epoll,FreeBSD上是kqueue,Mac OS X上是select)。有了gevent,协程的使用将无比简单,你根本无须像greenlet一样显式的切换,每当一个协程阻塞时,程序将自动调度,gevent处理了所有的底层细节。让我们看个例子来感受下吧。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
import gevent def test1(): print 12 gevent.sleep(0) print 34 def test2(): print 56 gevent.sleep(0) print 78 gevent.joinall([ gevent.spawn(test1), gevent.spawn(test2), ]) |
解释下,”gevent.spawn()”方法会创建一个新的greenlet协程对象,并运行它。”gevent.joinall()”方法会等待所有传入的greenlet协程运行结束后再退出,这个方法可以接受一个”timeout”参数来设置超时时间,单位是秒。运行上面的程序,执行顺序如下:
所以,程序运行下来的输出就是:
|
1 2 3 4 |
12 56 34 78 |
注意,这里与上一篇greenlet中第一个例子运行的结果不一样,greenlet一个协程运行完后,必须显式切换,不然会返回其父协程。而在gevent中,一个协程运行完后,它会自动调度那些未完成的协程。
我们换一个更有意义的例子:
|
1 2 3 4 5 6 7 8 |
import gevent import socket urls = ['www.baidu.com', 'www.gevent.org', 'www.python.org'] jobs = [gevent.spawn(socket.gethostbyname, url) for url in urls] gevent.joinall(jobs, timeout=5) print [job.value for job in jobs] |
我们通过协程分别获取三个网站的IP地址,由于打开远程地址会引起IO阻塞,所以gevent会自动调度不同的协程。另外,我们可以通过协程对象的”value”属性,来获取协程函数的返回值。
细心的朋友们在运行上面例子时会发现,其实程序运行的时间同不用协程是一样的,是三个网站打开时间的总和。可是理论上协程是非阻塞的,那运行时间应该等于最长的那个网站打开时间呀?其实这是因为Python标准库里的socket是阻塞式的,DNS解析无法并发,包括像urllib库也一样,所以这种情况下用协程完全没意义。那怎么办?
一种方法是使用gevent下的socket模块,我们可以通过”from gevent import socket”来导入。不过更常用的方法是使用猴子布丁(Monkey patching):
|
1 2 3 4 5 6 7 8 9 |
from gevent import monkey; monkey.patch_socket() import gevent import socket urls = ['www.baidu.com', 'www.gevent.org', 'www.python.org'] jobs = [gevent.spawn(socket.gethostbyname, url) for url in urls] gevent.joinall(jobs, timeout=5) print [job.value for job in jobs] |
上述代码的第一行就是对socket标准库打上猴子补丁,此后socket标准库中的类和方法都会被替换成非阻塞式的,所有其他的代码都不用修改,这样协程的效率就真正体现出来了。Python中其它标准库也存在阻塞的情况,gevent提供了”monkey.patch_all()”方法将所有标准库都替换。
|
1 |
from gevent import monkey; monkey.patch_all() |
使用猴子补丁褒贬不一,但是官网上还是建议使用”patch_all()”,而且在程序的第一行就执行。
协程状态有已启动和已停止,分别可以用协程对象的”started”属性和”ready()”方法来判断。对于已停止的协程,可以用”successful()”方法来判断其是否成功运行且没抛异常。如果协程执行完有返回值,可以通过”value”属性来获取。另外,greenlet协程运行过程中发生的异常是不会被抛出到协程外的,因此需要用协程对象的”exception”属性来获取协程中的异常。下面的例子很好的演示了各种方法和属性的使用。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
#coding:utf8 import gevent def win(): return 'You win!' def fail(): raise Exception('You failed!') winner = gevent.spawn(win) loser = gevent.spawn(fail) print winner.started # True print loser.started # True # 在Greenlet中发生的异常,不会被抛到Greenlet外面。 # 控制台会打出Stacktrace,但程序不会停止 try: gevent.joinall([winner, loser]) except Exception as e: # 这段永远不会被执行 print 'This will never be reached' print winner.ready() # True print loser.ready() # True print winner.value # 'You win!' print loser.value # None print winner.successful() # True print loser.successful() # False # 这里可以通过raise loser.exception 或 loser.get() # 来将协程中的异常抛出 print loser.exception |
之前我们讲过在”gevent.joinall()”方法中可以传入timeout参数来设置超时,我们也可以在全局范围内设置超时时间:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
import gevent from gevent import Timeout timeout = Timeout(2) # 2 seconds timeout.start() def wait(): gevent.sleep(10) try: gevent.spawn(wait).join() except Timeout: print('Could not complete') |
上例中,我们将超时设为2秒,此后所有协程的运行,如果超过两秒就会抛出”Timeout”异常。我们也可以将超时设置在with语句内,这样该设置只在with语句块中有效:
|
1 2 |
with Timeout(1): gevent.sleep(10) |
此外,我们可以指定超时所抛出的异常,来替换默认的”Timeout”异常。比如下例中超时就会抛出我们自定义的”TooLong”异常。
|
1 2 3 4 5 |
class TooLong(Exception): pass with Timeout(1, TooLong): gevent.sleep(10) |
greenlet协程间的异步通讯可以使用事件(Event)对象。该对象的”wait()”方法可以阻塞当前协程,而”set()”方法可以唤醒之前阻塞的协程。在下面的例子中,5个waiter协程都会等待事件evt,当setter协程在3秒后设置evt事件,所有的waiter协程即被唤醒。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
#coding:utf8 import gevent from gevent.event import Event evt = Event() def setter(): print 'Wait for me' gevent.sleep(3) # 3秒后唤醒所有在evt上等待的协程 print "Ok, I'm done" evt.set() # 唤醒 def waiter(): print "I'll wait for you" evt.wait() # 等待 print 'Finish waiting' gevent.joinall([ gevent.spawn(setter), gevent.spawn(waiter), gevent.spawn(waiter), gevent.spawn(waiter), gevent.spawn(waiter), gevent.spawn(waiter) ]) |
除了Event事件外,gevent还提供了AsyncResult事件,它可以在唤醒时传递消息。让我们将上例中的setter和waiter作如下改动:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
from gevent.event import AsyncResult aevt = AsyncResult() def setter(): print 'Wait for me' gevent.sleep(3) # 3秒后唤醒所有在evt上等待的协程 print "Ok, I'm done" aevt.set('Hello!') # 唤醒,并传递消息 def waiter(): print("I'll wait for you") message = aevt.get() # 等待,并在唤醒时获取消息 print 'Got wake up message: %s' % message |
队列Queue的概念相信大家都知道,我们可以用它的put和get方法来存取队列中的元素。gevent的队列对象可以让greenlet协程之间安全的访问。运行下面的程序,你会看到3个消费者会分别消费队列中的产品,且消费过的产品不会被另一个消费者再取到:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
import gevent from gevent.queue import Queue products = Queue() def consumer(name): while not products.empty(): print '%s got product %s' % (name, products.get()) gevent.sleep(0) print '%s Quit' def producer(): for i in xrange(1, 10): products.put(i) gevent.joinall([ gevent.spawn(producer), gevent.spawn(consumer, 'steve'), gevent.spawn(consumer, 'john'), gevent.spawn(consumer, 'nancy'), ]) |
put和get方法都是阻塞式的,它们都有非阻塞的版本:put_nowait和get_nowait。如果调用get方法时队列为空,则抛出”gevent.queue.Empty”异常。
信号量可以用来限制协程并发的个数。它有两个方法,acquire和release。顾名思义,acquire就是获取信号量,而release就是释放。当所有信号量都已被获取,那剩余的协程就只能等待任一协程释放信号量后才能得以运行:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
import gevent from gevent.coros import BoundedSemaphore sem = BoundedSemaphore(2) def worker(n): sem.acquire() print('Worker %i acquired semaphore' % n) gevent.sleep(0) sem.release() print('Worker %i released semaphore' % n) gevent.joinall([gevent.spawn(worker, i) for i in xrange(0, 6)]) |
上面的例子中,我们初始化了”BoundedSemaphore”信号量,并将其个数定为2。所以同一个时间,只能有两个worker协程被调度。程序运行后的结果如下:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
Worker 0 acquired semaphore Worker 1 acquired semaphore Worker 0 released semaphore Worker 1 released semaphore Worker 2 acquired semaphore Worker 3 acquired semaphore Worker 2 released semaphore Worker 3 released semaphore Worker 4 acquired semaphore Worker 4 released semaphore Worker 5 acquired semaphore Worker 5 released semaphore |
如果信号量个数为1,那就等同于同步锁。
同线程类似,协程也有本地变量,也就是只在当前协程内可被访问的变量:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
import gevent from gevent.local import local data = local() def f1(): data.x = 1 print data.x def f2(): try: print data.x except AttributeError: print 'x is not visible' gevent.joinall([ gevent.spawn(f1), gevent.spawn(f2) ]) |
通过将变量存放在local对象中,即可将其的作用域限制在当前协程内,当其他协程要访问该变量时,就会抛出异常。不同协程间可以有重名的本地变量,而且互相不影响。因为协程本地变量的实现,就是将其存放在以的”greenlet.getcurrent()”的返回为键值的私有的命名空间内。
讲到这里,大家肯定很想看一个gevent的实际应用吧,这里有一个简单的聊天室程序,基于Flask实现,大家可以参考下。
RTL8192EU(天猫魔盘)的无线网卡,使用如下方式安装驱动:|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
$ sudo apt-get install git linux-headers-generic build-essential dkms $ sudo apt-get install libelf-dev $ git clone https://github.com/wangqiang1588/rtl8192eu-linux-driver.git $ cd rtl8192eu-linux-driver $ make $ sudo make install $ sudo dkms remove rtl8192eu/1.0 --all $ sudo dkms add . $ sudo dkms install rtl8192eu/1.0 $ sudo reboot |
需求:将形如’y\xcc\xa6\xbb’的byte字符串转化为integer或者string
struct包|
1 2 |
import struct struct.unpack("<L", "y\xcc\xa6\xbb")[0] |
python3.2及以上若byte串采取大端法:
|
1 |
int.from_bytes(b'y\xcc\xa6\xbb', byteorder='big') |
若采取小端法,则:
|
1 |
int.from_bytes(b'y\xcc\xa6\xbb', byteorder='little') |
大端法:
|
1 2 |
s = 'y\xcc\xa6\xbb' num = int(s.encode('hex'), 16) |
小端法:
|
1 |
int(''.join(reversed(s)).encode('hex'), 16) |
array包|
1 2 |
import array integerValue = array.array("I", 'y\xcc\xa6\xbb')[0] |
其中I用于表示大端或小端,且使用此方法要注意自己使用的python版本。
如:
|
1 |
sum(ord(c) << (i * 8) for i, c in enumerate('y\xcc\xa6\xbb'[::-1])) |
又如:
|
1 2 3 4 5 6 |
def bytes2int( tb, order='big'): if order == 'big': seq=[0,1,2,3] elif order == 'little': seq=[3,2,1,0] i = 0 for j in seq: i = (i<<8)+tb[j] return i |
|
1 2 3 |
>>> import array >>> array.array('B', [17, 24, 121, 1, 12, 222, 34, 76]).tostring() '\x11\x18y\x01\x0c\xde"L' |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
import time import RPi.GPIO as GPIO GPIO.setmode(GPIO.BCM) btn_input = 4; LED_output = 17; # GPIO btn_input set up as input. GPIO.setup(btn_input, GPIO.IN) GPIO.setup(LED_output, GPIO.OUT) # handle the button event def buttonEventHandler_rising (pin): # turn LED on GPIO.output(LED_output,True) def buttonEventHandler_falling (pin): # turn LED off GPIO.output(LED_output,False) GPIO.add_event_detect(btn_input, GPIO.RISING, callback=buttonEventHandler_rising) GPIO.add_event_detect(btn_input, GPIO.FALLING, callback=buttonEventHandler_falling) try: while True : time.sleep(0.1) finally: GPIO.remove_event_detect(btn_input) GPIO.cleanup() |
树莓派Zero W,有两款小型号,一款是Raspberry Pi Zero W,另一款是Raspberry Pi Zero WH,两者的区别是一个出厂的时候没有焊接排针,另一款焊接了排针。W是Wireless的缩写,WH是Wireless With Head的缩写。
树莓派Zero W/WH(Raspberry Pi Zero W/WH) GPIO针脚定义如下图:
继续阅读树莓派Zero W/WH(Raspberry Pi Zero W/WH) GPIO针脚定义
有时需要用Matlab调试某些C语言开发的函数库,需要在Matlab里面查看执行效果。
整个的参考例子如下:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 |
#include <mex.h> // Check if some command is really some givent one static bool commandIs(const mxArray* mxCommand, const char* command) { double result; mxArray* plhs1[1]; mxArray* prhs1[1]; mxArray* plhs2[1]; mxArray* prhs2[2]; if (mxCommand == NULL) { mexErrMsgTxt("'mxCommand' is null"); return false; } if (command == NULL) { mexErrMsgTxt("'command' is null"); return false; } if (!mxIsChar(mxCommand)) { mexErrMsgTxt("'mxCommand' is not a string"); return false; } // First trim prhs1[0] = (mxArray*)mxCommand; mexCallMATLAB(1, plhs1, 1, prhs1, "strtrim"); // Then compare prhs2[0] = mxCreateString(command); prhs2[1] = plhs1[0]; mexCallMATLAB(1, plhs2, 2, prhs2, "strcmpi"); // Return comparison result result = mxGetScalar(plhs2[0]); return (result != 0.0); } static void processHelpMessageCommand(void) { mexPrintf("DspMgr('init') init return Handle,return nil if failed. use 'release' free memory\n"); mexPrintf("DspMgr('release',handle) free memory\n"); } static void processInitCommand(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) { char* example_buffer = malloc(512); plhs[0] = mxCreateNumericMatrix(1,1,mxUINT64_CLASS,mxREAL); long long *ip = (long long *) mxGetData(plhs[0]); *ip = (long long)example_buffer; } static void processReleaseCommand(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) { if(nrhs != 2) { mexErrMsgTxt("release need 1 params"); } else { if(!mxIsUint64(prhs[1])) { mexErrMsgTxt("release handle must be UINT64 format"); return; } int M=mxGetM(prhs[1]); //获得矩阵的行数 int N=mxGetN(prhs[1]); //获得矩阵的列数 if((1 != M) &&(1 != N)) { mexErrMsgTxt("release handle must be 1*1 array format"); return; } long long ip = mxGetScalar(prhs[1]); char* example_buffer = (char*)ip; free(example_buffer); //return true avoid warnning plhs[0] = mxCreateNumericMatrix(1,1,mxINT8_CLASS,mxREAL); char* mx_data = (char *) mxGetData(plhs[0]); mx_data[0] = 1; } } // Mex entry point void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) { // Arguments parsing if (nrhs < 1) { mexErrMsgTxt("Not enough input arguments. use 'DspMgr help' for help message."); return; } if (!mxIsChar(prhs[0])) { mexErrMsgTxt("First parameter must be a string."); return; } // Command selection if (commandIs(prhs[0], "HELP")) { processHelpMessageCommand(); } else if (commandIs(prhs[0], "init")) { processInitCommand(nlhs, plhs, nrhs, prhs); } else if (commandIs(prhs[0], "release")) { processReleaseCommand(nlhs, plhs, nrhs, prhs); } else { mexErrMsgTxt("Unknown command or command not implemented yet."); } } |
尤其注意上面例子里我们如何隐藏一个C里申请的指针并传递给Matlab。
Matlab的调用例子如下:
|
1 2 3 4 5 6 |
mex -output DspMgr 'CFLAGS="\$CFLAGS -std=c99"' '*.c' v = DspMgr('init') DspMgr('release',v) |
Telnet server using gevent or threading.
Copied from http://pytelnetsrvlib.sourceforge.net/ and modified to support gevent, better input handling, clean asynchronous messages and much more. Licensed under the LGPL, as per the SourceForge notes.
This library allows you to easily create a Telnet server, powered by your Python code. The library negotiates with a Telnet client, parses commands, provides an automated help command, optionally provides login queries, then allows you to define your own commands.
You use the library to create your own handler, then pass that handler to a StreamServer or TCPServer to perform the actual connection tasks.
This library includes two flavors of the server handler, one uses separate threads, the other uses greenlets (green pseudo-threads) via gevent.
The threaded version uses a separate thread to process the input buffer and semaphores reading and writing. The provided test server only handles a single connection at a time.
The green version moves the input buffer processing into a greenlet to allow cooperative multi-processing. This results in significantly less memory usage and nearly no idle processing. The provided test server handles a large number of connections.
telnetsrv is available through the Cheeseshop. You can use easy_install or pip to perform the installation.
|
1 2 3 |
$ sudo easy_install telnetsrv $ sudo easy_install gevent |
or
|
1 2 3 |
$ sudo pip install telnetsrv $ sudo pip install gevent |
Note that there are no dependancies defined, but if you want to use the green version, you must also install gevent.
Import the TelnetHandler base class and command function decorator from either the green class or threaded class, then subclass TelnetHandler to add your own commands which are methods decorated with @command.
|
1 2 3 |
from telnetsrv.threaded import TelnetHandler, command class MyHandler(TelnetHandler): ... |
|
1 2 3 4 |
from telnetsrv.green import TelnetHandler, command class MyHandler(TelnetHandler): ... |
Commands can be defined by using the command function decorator.
|
1 2 3 |
@command('echo') def command_echo(self, params): ... |
Commands can also be defined by prefixing any method with “cmd”. For example, this also creates an echocommand:
|
1 2 |
def cmdECHO(self, params): ... |
This method is less flexible and may not be supported in future versions.
Any command parameters will be passed to this function automatically. The parameters are contained in a list. The user input is parsed similar to the way Bash parses text: space delimited, quoted parameters are kept together and default behavior can be modified with the \ character. If you need to access the raw text input, inspect the self.input.raw variable.
|
1 |
Telnet Server> echo 1 "2 3" |
|
1 2 |
params == ['1', '2 3'] self.input.raw == 'echo 1 "2 3"\n' |
|
1 2 3 4 |
Telnet Server> echo 1 \ ... 2 "3 ... 4" "5\ ... 6" |
|
1 |
params == ['1', '2', '3\n4', '56'] |
|
1 |
Telnet Server> echo 1\ 2 |
|
1 |
params == ['1 2'] |
The command’s docstring is used for generating the console help information, and must be formatted with at least 3 lines:
If there is no line 2, line 1 will be used for the long description as well.
|
1 2 3 4 5 6 7 8 |
@command('echo') def command_echo(self, params): '''<text to echo> Echo text back to the console. This command simply echos the provided text back to the console. ''' pass |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
Telnet Server> help ? [<command>] - Display help BYE - Exit the command shell ECHO <text to echo> - Echo text back to the console. ... Telnet Server> help echo ECHO <text to echo> This command simply echos the provided text back to the console. Telnet Server> |
To create an alias for the new command, set the method’s name to a list:
|
1 2 3 |
@command(['echo', 'copy']) def command_echo(self, params): ... |
The decorator may be stacked, which adds each list to the aliases:
|
1 2 3 4 5 |
@command('echo') @command(['copy', 'repeat']) @command('ditto') def command_echo(self, params): ... |
To hide the command (and any alias for that command) from the help text output, pass in hidden=True to the decorator:
|
1 2 3 |
@command('echo', hidden=True) def command_echo(self, params): ... |
The command will not show when the user invokes help by itself, but the detailed help text will show if the user invokes help echo.
When stacking decorators, any one of the stack may define the hidden parameter to hide the command.
These will be provided for inspection.
|
1 2 3 4 5 6 7 8 9 |
@command('info') def command_info(self, params): ''' Provides some information about the current terminal. ''' self.writeresponse( "Username: %s, terminal type: %s" % (self.username, self.TERM) ) self.writeresponse( "Command history:" ) for c in self.history: self.writeresponse(" %r" % c) |
Lower level functions:
self.writeline( TEXT )
self.write( TEXT )
Higher level functions:
self.writemessage( TEXT ) - for clean, asynchronous writing. Any interrupted input is rebuilt.
self.writeresponse( TEXT ) - to emit a line of expected output
self.writeerror( TEXT ) - to emit error messages
The writemessage method is intended to send messages to the console without interrupting any current input. If the user has entered text at the prompt, the prompt and text will be seamlessly regenerated following the message. It is ideal for asynchronous messages that aren’t generated from the direct user input.
self.readline( prompt=TEXT )
Setting the prompt is important to recreate the user input following a writemessage interruption.
When requesting sensative information from the user (such as requesting a password) the input should not be shown nor should it have access to or be written to the command history. readline accepts two optional parameters to control this, echo and user_history.
self.readline( prompt=TEXT, echo=False, use_history=False )
Override these class members to change the handler’s behavior.
Default: "You have connected to the telnet server."
Default: pass
Default: pass
Default: None
Default: False
Default: False
If you want to change how the output is displayed, override one or all of the write classes. Make sure you call back to the base class when doing so. This is a good way to provide color to your console by using ANSI color commands. See http://en.wikipedia.org/wiki/ANSI_escape_code
|
1 2 3 |
def writeerror(self, text): '''Write errors in red''' TelnetHandler.writeerror(self, "\x1b[91m%s\x1b[0m" % text ) |
Now you have a shiny new handler class, but it doesn’t serve itself - it must be called from an appropriate server. The server will create an instance of the TelnetHandler class for each new connection. The handler class will work with either a gevent StreamServer instance (for the green version) or with a SocketServer.TCPServer instance (for the threaded version).
|
1 2 3 4 5 6 |
import SocketServer class TelnetServer(SocketServer.TCPServer): allow_reuse_address = True server = TelnetServer(("0.0.0.0", 8023), MyHandler) server.serve_forever() |
The TelnetHandler class includes a streamserver_handle class method to translate the required fields from a StreamServer, allowing use with the gevent StreamServer (and possibly others).
|
1 2 3 |
import gevent.server server = gevent.server.StreamServer(("", 8023), MyHandler.streamserver_handle) server.server_forever() |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
import gevent, gevent.server from telnetsrv.green import TelnetHandler, command class MyTelnetHandler(TelnetHandler): WELCOME = "Welcome to my server." @command(['echo', 'copy', 'repeat']) def command_echo(self, params): '''<text to echo> Echo text back to the console. ''' self.writeresponse( ' '.join(params) ) @command('timer') def command_timer(self, params): '''<time> <message> In <time> seconds, display <message>. Send a message after a delay. <time> is in seconds. If <message> is more than one word, quotes are required. example: > TIMER 5 "hello world!" ''' try: timestr, message = params[:2] time = int(timestr) except ValueError: self.writeerror( "Need both a time and a message" ) return self.writeresponse("Waiting %d seconds...", time) gevent.spawn_later(time, self.writemessage, message) server = gevent.server.StreamServer(("", 8023), MyTelnetHandler.streamserver_handle) server.serve_forever() |
If the paramiko library is installed, the TelnetHanlder can be used via an SSH server for significantly improved security. paramiko_ssh contains SSHHandler and getRsaKeyFile to make setting up the server trivial. Since the authentication is done prior to invoking the TelnetHandler, any authCallback defined in the TelnetHandler is ignored.
If using the green version of the TelnetHandler, you must use Gevent’s monkey patch_all prior to importing from paramiko_ssh.
|
1 2 |
from gevent import monkey; monkey.patch_all() from telnetsrv.paramiko_ssh import SSHHandler, getRsaKeyFile |
The SocketServer/StreamServer sets up the socket then passes that to an SSHHandler class which authenticates then starts the SSH transport. Within the SSH transport, the client requests a PTY channel (and possibly other channel types, which are denied) and the SSHHandler sets up a TelnetHandler class as the PTY for the channel. If the client never requests a PTY channel, the transport will disconnect after a timeout.
To thwart man-in-the-middle attacks, every SSH server provides an RSA key as a unique fingerprint. This unique key should never change, and should be stored in a local file or a database. The getRsaKeyFilemakes this easy by reading the given key file if it exists, or creating the key if it does not. The result should be read once and set in the class definition.
Easy way:
Long way:
|
1 2 3 4 5 6 7 8 9 |
from paramiko_ssh import RSAKey # Make a new key - should only be done once per server during setup new_key = RSAKey.generate(1024) save_to_my_database( 'server_fingerprint', str(new_key) ) ... host_key = RSAKey( data=get_from_my_database('server_fingerprint') ) |
Users can authenticate with just a username, a username/publickey or a username/password. Up to three callbacks can be defined, and if all three are defined, all three will be tried before denying the authentication attempt. An SSH client will always provide a username. If no authCallbackXX is defined, the SSH authentication will be set to “none” and any username will be able to log in.
If defined, this is always tried first.
Default: None
Default: None
If defined, this is always tried last.
Default: None
SSHHandler uses Paramiko’s ServerInterface as one of its base classes. If you are familiar with Paramiko, feel free to instead override the authentication callbacks as needed.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
from gevent import monkey; monkey.patch_all() import gevent.server from telnetsrv.paramiko_ssh import SSHHandler, getRsaKeyFile from telnetsrv.green import TelnetHandler, command class MyTelnetHandler(TelnetHandler): WELCOME = "Welcome to my server." @command(['echo', 'copy', 'repeat']) def command_echo(self, params): '''<text to echo> Echo text back to the console. ''' self.writeresponse( ' '.join(params) ) class MySSHHandler(SSHHandler): # Set the unique host key host_key = getRsaKeyFile('server_fingerprint.key') # Instruct this SSH handler to use MyTelnetHandler for any PTY connections telnet_handler = MyTelnetHandler def authCallbackUsername(self, username): # These users do not require a password if username not in ['john', 'eric', 'terry', 'graham']: raise RuntimeError('Not a Python!') def authCallback(self, username, password): # Super secret password: if password != 'concord': raise RuntimeError('Wrong password!') # Start a telnet server for just the localhost on port 8023. (Will not request any authentication.) telnetserver = gevent.server.StreamServer(('127.0.0.1', 8023), MyTelnetHandler.streamserver_handle) telnetserver.start() # Start an SSH server for any local or remote host on port 8022 sshserver = gevent.server.StreamServer(("", 8022), MySSHHandler.streamserver_handle) sshserver.serve_forever() |
See https://github.com/ianepperson/telnetsrvlib/blob/master/test.py
最近在配置Raspberry Pi Zero W,使用的系统为2018-06-27-raspbian-stretch-lite,我们的需求是把这台Raspberry Pi Zero W配置为开放Wi-Fi模式的AP。
Raspberry Pi Zero W只有一块无线网卡,如果被配置成AP模式不是太好操作,可以通过预留的Macro USB接口外接一个USB有线网卡来实现远程访问,当然也可以直接接入显示器,USB Hub外接鼠标键盘操作。
执行如下脚本配置:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
#目前测试发现udhcpd在标准树莓派上能正常工作,但是在Pi Zero W上,重启之后, #不能正常分配IP,应该是服务在无线网卡没有初始化完成就已经启动导致的, #我们使用dnsmasq替代后可以正常工作 $ sudo apt-get -y remove udhcpd $ sudo apt-get -y install hostapd dnsmasq #备份配置文件 $ sudo cp /etc/dnsmasq.conf /etc/dnsmasq.conf.bak #配置分配的IP段 $ sudo sed -i "s/^#dhcp-range=192.168.0.50,192.168.0.150,12h/dhcp-range=192.168.0.50,192.168.0.150,12h/g" /etc/dnsmasq.conf #AP名字 $ export AP_NAME="AP" #为无线网卡配置静态IP $ export WLAN_IP=192.168.0.1 $ sudo ifconfig wlan0 $WLAN_IP $ sudo sed -i '$a\interface wlan0' /etc/dhcpcd.conf $ echo "static ip_address=${WLAN_IP}/24" | sudo tee -a /etc/dhcpcd.conf #hostapd配置,配置为开放模式 $ sudo touch /etc/hostapd/hostapd.conf $ echo "interface=wlan0" | sudo tee -a /etc/hostapd/hostapd.conf $ echo "ssid=${AP_NAME}" | sudo tee -a /etc/hostapd/hostapd.conf #WiFi工作的频段 1-13 $ echo "channel=9" | sudo tee -a /etc/hostapd/hostapd.conf #硬件工作模式 g simply means 2.4GHz $ echo "hw_mode=g" | sudo tee -a /etc/hostapd/hostapd.conf #验证方式为开放模式 1=wpa, 2=wep, 3=both $ echo "auth_algs=1" | sudo tee -a /etc/hostapd/hostapd.conf # 802.11n support $ echo "ieee80211n=1" | sudo tee -a /etc/hostapd/hostapd.conf #备份配置文件 $ sudo cp /etc/default/hostapd /etc/default/hostapd.bak #修改配置文件 $ sudo sed -i "s/#DAEMON_CONF=\"\"/DAEMON_CONF=\"\/etc\/hostapd\/hostapd.conf\"/g" /etc/default/hostapd #启动networking和hostapd服务,注意先后顺序,先使用networking服务设置IP,再更新hostapd $ sudo service networking restart $ sudo service hostapd restart $ sudo service dnsmasq restart #设置开机启动 $ sudo update-rc.d hostapd enable #重启设备,检查配置是否已经生效 $ sudo reboot |
最近在使用Raspberry Pi Zero W,在创建系统镜像的时候,使用如下命令,发现非常缓慢,时间往往以小时计算:
|
1 2 3 |
$ diskutil unmountDisk /dev/disk2 $ sudo dd if=~/Downloads/2018-06-27-raspbian-stretch-lite.img of=/dev/disk2 |
如果要解决这个问题,那么可以使用如下方式:
|
1 2 3 |
$ diskutil unmountDisk /dev/rdisk2 $ sudo dd if=~/Downloads/2018-06-27-raspbian-stretch-lite.img of=/dev/rdisk2 bs=1m |
注意两个命令的区别,一个是 /dev/disk2 ,一个是 /dev/rdisk2 , 两者的区别可以通过如下命令来查看:
|
1 |
$ man hdiutil |
可以看到如下介绍:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
.......................... DEVICE SPECIAL FILES Since any /dev entry can be treated as a raw disk image, it is worth not- ing which devices can be accessed when and how. /dev/rdisk nodes are character-special devices, but are "raw" in the BSD sense and force block-aligned I/O. They are closer to the physical disk than the buffer cache. /dev/disk nodes, on the other hand, are buffered block-special devices and are used primarily by the kernel's filesystem code. It is not possible to read from a /dev/disk node while a filesystem is mounted from it, but anyone with read access to the appropriate /dev/rdisk node can use hdiutil verbs such as fsid or pmap with it. Beware that information read from a raw device while a filesystem is mounted may not be consistent because the consistent data is stored in memory or in the filesystem's journal. The DiskImages framework will attempt to use authopen(1) to open any device which it can't open (due to EACCES) for reading with open(2). Depending on session characteristics, this behavior can cause apparent hangs while trying to access /dev entries while logged in remotely (an authorization panel is waiting on console). Generally, the /dev/disk node is preferred for imaging devices (e.g. convert or create -srcdevice operations), while /dev/rdisk is usable for the quick pmap or fsid. In particular, converting the blocks of a mounted journaled filesystem to a read-only image will prevent the volume in the image from mounting (the journal will be permanently dirty). ................................ |
根据介绍,rdisk属于原始设备(rawdisk),不必经过操作系统的文件系统缓冲处理,相当于直接操作硬件,速度非常快。但是像macOS High Sierra这种出现20x速度差别的情况,就不是太好理解了。
后面 bs=1m 参数也很重要,要求拷贝写入的时候整块 (1MB) 写入(否则是逐个字节操作,写入次数非常多,性能很差),这样才能起到加速作用。