Tags » Gevent

How we use gevent to go fast

“Not too long ago we were tackling the challenge of fixing a legacy Python system and converting a two-year old single-threaded codebase with hundreds of thousands of lines of code to a multi-threaded codebase. 55 more words

Programming

关于gevent的问题

import gevent.monkey
gevent.monkey.patch_socket()
 
import gevent
import urllib2
import simplejson as json

from gevent import queue
from gevent import pool
 
def fetch(pid):
    response = urllib2.urlopen('http://json-time.appspot.com/time.json')
    result = response.read()
    json_result = json.loads(result)
    datetime = json_result['datetime']
 
    print 'Process ', pid, datetime
    return json_result['datetime']

# worker_sequence = []
def asynchronous(worker_sequence):
    for i in range(1,10):
        g = gevent.spawn(fetch, i)
        worker_sequence.append(g)
    gevent.joinall(worker_sequence)
    print len(worker_sequence)

# worker_sequence = gevent.queue.Queue()
# or, worker_sequence = gevent.queue.JoinableQueue()
# error: "this operation would block forever" after all works are done 
def asynchronous(worker_sequence):
    for i in range(1,10):
        g = gevent.spawn(fetch, i)
        worker_sequence.put(g)
    gevent.joinall(worker_sequence)

# worker_poll = gevent.pool.Pool()
def asynchronous(worker_pool):
    for i in range(1,10):
        g = gevent.spawn(fetch, i)
        worker_pool.add(g)
    print len(worker_pool)
    gevent.joinall(worker_pool)
    print len(worker_pool)

threads = []
#threads = queue.JoinableQueue()
#threads = queue.Queue()
#threads = pool.Pool(100)

print 'Asynchronous:'
asynchronous(threads)
… 35 more words

Tornado vs. Gevent - Benchmarks

“After FriendFeed was acquired several years ago, Tornado stagnated a bit, but this past year the community has been hard at work. Version 2.0 came out this past June, and the framework is now at 2.4.1 (at the time of this writing). 60 more words

Programming

GreenletProfiler, A Fast Python Profiler For Gevent

“If you use Gevent, you know it’s great for concurrency, but alas, none of the Python performance profilers work on Gevent applications. So I’m taking matters into my own hands. 27 more words

Programming

Introduction to Gevent

“In a previous post, I described how to build a web chat server with socket.io and gevent. If you’re trying to actually learn gevent, socket.io, however, it’s probably not the best place to start. 56 more words

Programming

Fast, Multipart Downloads from S3

Using the “Range” header and greenlets, you can do very fast downloads from S3. The speed improvements are considerably higher than with uploads. With a 130M file, it was more than three-times faster to parallelize a download. 71 more words

Tools/Services

Doing Fast Multipart Uploads to S3 Using Greenlets

S3 allows you to upload pieces of large files in parallel. Unfortunately, most/all of the examples that I’ve seen online are inefficient or inconvenient. For example: 770 more words

Tools/Services