Imported a number of patches from stdlib by Antoine Pitrou:
Calling makefile() method on an SSL object would prevent the underlying socket from being closed until all objects get truely destroyed (Python issue #5238).
SSL handshake would ignore the socket timeout and block indefinitely if the other end didn’t respond (Python issue #5103).
When calling getpeername() in SSLSocket.__init__, only silence exceptions caused by the “socket not connected” condition.
Added support for ciphers argument.
Updated SSLSocket.send and SSLSocket.recv methods to match the behavior of stdlib ssl better.
Fixed ssl.SSLObject to delete events used by other greenlets when closing the instance (issue #34).
Miscellaneous:
Made BaseServer accept long values as pool argument in addition to int.
Made http._requests attribute public.
Updated webchat example to use file on disk rather than in-memory sqlite database to avoid OperationalError.
Fixed webproxy.py example to be runnable under external WSGI server.
Fixed bogus failure in test__exc_info.py.
Added new test to check PEP8 conformance: xtest_pep8.py.
Fixed BackdoorServer close the connection on SystemExit and simplified the code.
Made Pool raise ValueError when initialized with size=0.
Updated setup.py--libevent to configure and make libevent if it’s not built already.
Updated setup.py to use setuptools if present and add dependency on greenlet.
Fixed doc/mysphinxext.py to work with Sphinx 1. Thanks by Örjan Persson.
Mondrian has been ported to Google App Engine: Rietveld. It's open source (Apache license), you can deploy it on your app-engine account, or port to vanilla Django and deploy on your own server.
My first project as a Google engineer was an internal web app for code review. According to Wikipedia, code review is "systematic examination (often as peer review) of computer source code intended to find and fix mistakes overlooked in the initial development phase, improving both the overall quality of software and the developers' skills." Not an exciting topic, perhaps, but the internal web app, which I code-named Mondrian after one of my favorite Dutch painters, was an overnight success among Google engineers (who evidently value software quality and skills development :-). I even gave a public presentation about it: you can watch the video on YouTube.
I've always hoped that we could release Mondrian as open source, but so far it hasn't happened: due to its popularity inside Google, it became more and more tied to proprietary Google infrastructure like Bigtable, and it remained limited to Perforce, the commercial revision control system most used at Google.
Fortunately, now that I work for the Google App Engine team, I've been able to write a new web app that incorporates many ideas (and even some code!) from Mondrian, and release it as open source. The Python open source community has been trying out Rietveld for the past few days, and has already been using it to do code reviews for Python (as well as providing valuable feedback in the form of bug reports and feature requests). Of course, the tool is not language-specific: you can use it for code reviews for any language!
Introducing Rietveld
To stick with the naming theme, I gave this new web app the code name Rietveld, after Gerrit Rietveld, one of my favorite Dutch architects and the designer of theZig-Zag chair. However, because most English speakers have trouble spelling his name correctly, the "live" web app is known simply as codereview.appspot.com.
The Rietveld app serves several purposes at once: it is a demo of fairly large-scale use of the popular web framework Django with App Engine, it makes some of the trickier (but portable) code we wrote for Mondrian available for reuse under the Apache 2.0 license, and it makes web-based code review available for many projects using Subversion repositories. Right now, any project hosted on Google Code can use Rietveld, as well as the Python subversion server. Support for arbitrary subversion servers is forthcoming.
While a public instance of Rietveld is running at codereview.appspot.com, organizations are of course free to run their own instance restricted and/or tailored to their own needs. That's what open source is for!
How Rietveld Manages Code Reviews
So what can you do with Rietveld? The basic workflow is:
Developer makes some changes in their Subversion workspace.
Developer uploads a patch in the form of svn diff output to Rietveld, using a small script named upload.py. This creates a new issue for them on the Rietveld website.
Developer goes to the issue that was just created on the Rietveld site, adds the email addresses of one or more reviewers, and causes Rietveld to send an email to the reviewer(s).
Reviewer navigates to the issue on the Rietveld site, browses the side-by-side diffs linked from there. A side-by-side diff shows the old and new version of the source code side by side, with deleted text on the left marked with a light red background, and inserted text on the right marked with a light green background. (Two different shades of red and green each are used, to highlight the differences at a finer-grain level than blocks of lines. This helps find one-character changes and clarifies diffs that just reflow a lot of text.)
Reviewer inserts inline comments directly into the side-by-side diffs, by double-clicking lines on which they want to comment. Inline comments are initially created in draft mode, which means that only the comment author can see (and edit) them.
Reviewer publishes comments, making them visible to everyone else, and sending an email to the developer (and to other reviewers) summarizing the inline comments with a little bit of context.
At this point, the developer can reply to inline comments directly on the Rietveld website using exactly the same mechanism as used by the reviewer. Replies simply become additional inline draft comments. The developer can also revise their code and upload a new version of the patch. The new version is attached to the same issue, and reviewers can choose to view the diffs afresh, or view the delta between the new and the old version of the patch. The latter feature is particularly helpful for large code reviews that require several iterations to reach agreement between developer and reviewer: the reviewer doesn't have to re-review stuff that didn't change between revisions and was already approved.
Coming Developments
I'm far from done with this application. Some features found in Mondrian that would be useful in Rietveld as well have not been implemented yet due to time constraints. The first users are also already asking for features I had never dreamed of, thanks to the many different styles of development found in the open source world. I have made the source code available as open source in this early stage in the hope to solicit outside contributions. My intention is to add outside developers to the project as soon as possible. As with Python, I am planning to remain in charge and review contributions carefully.
配置文件.顾名思议就是存放配置的文件.下面是个例子 [info] age = 21 name = chen sex = male
其中[ ] 中的info是这段配置的名字 下面age,name都是属性
下面的代码演示了如何读取配置文件.和修改配置中变量的值
from__future__import with_statement import ConfigParser config=ConfigParser.ConfigParser() with open('testcfg.cfg','rw') as cfgfile: config.readfp(cfgfile) name=config.get('info','name') age=config.get('info','age') print name print age config.set('info','sex','male') config.set('info','age','21') age=config.get('info','age') print name print age
• Able to handle a variety of jobs at once • Flexibility • Attention to detail • Have a flexible and willing attitude • Work unsupervised • Excellent work history • Communication skills
How-To: Spawn PHP with supervisord (for NGINX on Debian)
Just to muddy the waters for people trying to decide between spawn-cgi and php-fpm, there's a third way to spawn php-cgi for NGINX. It's a process manager called supervisord, which will start any number of processes you want (you can use it for ssh, mysql, etc.) -- monitor them while they run, and restart them if they fail. I was previously using php-fpm, so that's reflected in some of the code below.
It can be slightly tricky, because it starts processes as sub-processes of itself. So any process you start with supervisord can't be running in daemon mode.
Socket needs to match the socket that NGINX is looking for. 127.0.0.1:9000 is pretty standard, but look at your nginx.conf file. The commented line is there if you want to listen on a unix socket instead.
I've been using supervisord to start all of the processes that I want monitored, so:
Then reboot, your VPS and eveything should be running. If you'd rather test it first, don't add ssh to your supervisord.conf file (I suppose you could get booted from your shell if anything goes haywire). Then:
If you go to the fancy web interface (set up above in the [supervisorctl] options), you should see what's running (hopefully everything) and what's not (hopefully nothing). Or you can run top or whatever command line utility you desire.
Now here's the scary part. Because every process is a sub-process of supervisord, if supervisord were to fail, it stands to reason that every process that it started will fail as well. So, and this might be overkill, I run monit as well. I use monit to watch supervisor. In monitrc:
Code:
check process supervisord
with pidfile /path/to/supervisord.pid
start program "/etc/init.d/supervisord start"
stop program "/etc/init.d/supervisord stop"
if failed unixsocket /tmp/supervisor.sock then restart
if 5 restarts within 5 cycles then timeout
If monit goes down, and then supervisor goes down, then I guess I'm screwed. But I don't anticipate that happening. If you're prone to paranoia, you could daisy-chain runit, god, nagios, and who knows what else.
The whole process may seem a bit convoluted, but once you get it running it's wonderful. At least I think so.
nginx / ngx_supervisord (project fully funded by megiteam.pl) Module providing nginx with API to communicate with supervisordand manage (start/stop) backends on-demand.
As a "side effect", it also provides a way for dynamically taking backend servers out of rotation.
superlance plugins for supervisor
=================================
Superlance is a package of plugin utilities for monitoring and
controlling processes that run under `supervisor
`_.
Currently, it provides two scripts:
``httpok`` -- This script can be used as a supervisor event listener
(subscribed to TICK events) which will restart a "hung" HTTP server
process, which is defined as a process in the RUNNING state which does
not respond in an appropriate or timely manner to an HTTP GET request.
``crashmail`` -- This script will email a user when a process enters
the EXITED state unexpectedly.
``memmon`` -- See the description below.
Memmon Overview
---------------
memmon is a supervisor "event listener" which may be subscribed to a
concrete TICK_x event. When memmon receives a TICK_x event (TICK_60 is
recommended, indicating activity every 60 seconds), memmon checks that a
configurable list of programs (or all programs running under supervisor) are
not exceeding a configurable about of memory (resident segment size, or RSS).
If one or more of these processes is consuming more than the amount of memory
that memmon believes it should, memmon will restart the process. memmon can be
configured to send an email notification when it restarts a process.
memmon is known to work on Linux and Mac OS X, but has not been tested on
other operating systems (it relies on ps output and command-line switches).
memmon is incapable of monitoring the process status of processes which are
not supervisord child processes.
Memmon Command
--------------
memmon is a "console script" installed when you install supervisor. Although
memmon is an executable program, it isn't useful as a general-purpose script:
it must be run as a supervisor event listener to do anything useful. memmon
accepts the following options:
I have looked around of documentation on this but haven't found any. Is
it possible to keep the nginx master process from going into the
background/damonizing when starting it? I would like to have it run in
the foreground rather than becoming a daemon.
Thanks.
-Joe
On Wednesday 07 May 2008, Joe Williams wrote:
> I have looked around of documentation on this but haven't found any. Is> it possible to keep the nginx master process from going into the> background/damonizing when starting it? I would like to have it run in> the foreground rather than becoming a daemon.
daemon off
Agile web development for businesses and creative firms Exceptional applications built using the Django web framework
We're Hiring!
We’re looking for a Django developer to join our team. With the impending departure of Armin Ronacher to finish his degree, we’re going to be at least one developer short this Fall. We’re looking for somebody who can jump right in and start contributing, primarily as a back-end Python/Django developer.
We’re a distributed company, so location does not matter. We like to hire managers of one and prefer people who are active in the Django open source community.
Initially, this will be a full-time (or as close as possible) contract position. If things work out, we’ve got a workload that can keep you around indefinitely. If you are interested, send an email to info@lincolnloop.com and include the following:
A while ago I found a solution to run Solr with supervisord, without leaking java processes, which you get when you use the generated solr-instance script with supervisord.
You need something like this in your supervisord.conf:
The key is the directory option. With that the command is executed in that directory, which starts Solr correctly and doesn’t leave java processes when stopping it with supervisord.
I really like supervisord for long-running process management. It is Python, so it is easy to install and it is easy to script with tools like Fabric. Lately I’ve been deploying smaller Django/WSGI sites with Green Unicorn behind Nginx proxied over a Unix socket which makes for a nice little setup on shared hosting and RAM starved VPSes.
After installing supervisord with something like sudo easy_install supervisor, you’ll want to make sure it runs after you reboot your machine (and gets restarted in case it crashes). If you are on a Linux system with upstart (and have root access), you can bypass the ugly init scripts and use this simple file placed in /etc/init/supervisord.conf to manage your supervisord process:
description "supervisord"
start on runlevel [2345]
stop on runlevel [!2345]
respawn
exec /usr/local/bin/supervisord --nodaemon --configuration /etc/supervisord.conf
Now that it is in upstart, you can start it with the command sudo start supervisord.
If you are on shared hosting, you probably won’t have permissions to do this, but if you have access tocron, you can make sure it gets started in case the server reboots by adding this to your cron jobs:
In the event supervisord crashes or is killed off by a sysadmin, you’re out of luck. If this happens, you’ll want to look into having a cron job periodically poll the process pid to see if it is still alive and restart it if it is not. I haven’t needed this (yet), so that is left as an exercise for the reader :)
I am using daemontools to monitor gunicorn. When I update my code I send gunicorn a SIGHUP to gracefully reload the code. This creates all sorts of problems. As expected gunicorn starts up a new process and then shuts the original process down. At which point daemontools tries to start gunicorn as it thinks gunicorn has died. I have a tempary hack which I’ll spare you from.
I have been looking for an alternative and supervisord is high on the list but I have not been able to find anything conclusive on sending signals to monitored processes. What is your experience in getting gunicorn to gracefully reload code with your setup?
2009-05-14, Thursday 17:42 PM | 
0 comments | 
0 pingbacks | 
supervisor, virtual host, webfaction, zine
How-To: Spawn PHP with supervisord (for NGINX on Debian)
I am using daemontools to monitor gunicorn. When I update my code I send gunicorn a SIGHUP to gracefully reload the code. This creates all sorts of problems. As expected gunicorn starts up a new process and then shuts the original process down. At which point daemontools tries to start gunicorn as it thinks gunicorn has died. I have a tempary hack which I’ll spare you from.
I have been looking for an alternative and supervisord is high on the list but I have not been able to find anything conclusive on sending signals to monitored processes. What is your experience in getting gunicorn to gracefully reload code with your setup?
@Simon: You could use the supervisorctl tool to signal supervisord to restart the process.
@ZeissS That would stop and start the process and not gracefully reload the code.
I guess my question is does supervisord get its knickers in a twist if I “kill -HUP `cat /tmp/gunicorn.pid`” like daemontools does?
Seeing that you can’t give supervisord a pid file for a process, I guess the answer is yes.