- Solar Energy: This is What a Disruptive Technology Looks Like (Brian McConnell) — In 1977, solar cells cost upwards of $70 per Watt of capacity. In 2013, that cost has dropped to $0.74 per Watt, a 100:1 improvement (source: The Economist). On average, solar power improves 14% per year in terms of energy production per dollar invested.
- Process Managers — overview of the tools that keep your software running.
- Bittorrent Sync — Dropbox-like features, BitTorrent under the hood.
- Brython — Python interpreter written in Javascript, suitable for embedding in webpages. (via Nelson Minar)
"Python" entries
Hadoop Training, OpenStreetMap Sprint, MakersFactory Kids’ Programming Camp, and More
Tech events you don't want to miss
Each Monday, we round up upcoming event highlights from the programming and technology spaces. Have an event to share? Send us a note.
Twisted Python: the engine of your Internet webcast: Jessica McKellar presents an architectural overview of the Python networking library, Twisted, and instructs on how to build robust clients and servers for popular and custom network protocols. Register for this free webcast.
Date: 10 a.m. PT, June 6 Location: Online webcast
2 Day Hadoop Training June 2013: This course offers a fast-paced technical overview of the Hadoop landscape, targeted toward both technical and non-technical people who want to understand the emerging world of big data. For more information and to register, visit the event page.
Date: June 8–9 Location: Sunnyvale, CA
Thinking in Code
Multithreading for your brain
Allen Downey, is a Professor of Computer Science at Olin College of Engineering. He has written three books for us, so far, Think Python, Think Stats, and Think Complexity. I recently got the chance to sit down with him to talk about how writing good code can and should actually change the way you think—and make you a better programmer.
- Think differently when dealing with natural language, math, and code [Discussed at 0:38]
- Better code equals a better brain [Discussed at 2:05]
- Look to Python for a jumpstart [Discussed at 4:43]
- Manage the complexity of code for long lasting programs [Discussed at 7:12]
- Dealing with legacy programs? Try to see the big picture. [Discussed at 8:40]
You can view the entire interview in the following video.
Four short links: 24 April 2013
Solar Numbers, Process Managers, BitTorrent Sync, and Motherfrickin' Snakes in Your Motherfrickin' Browser
Twisted Python: The engine of your Internet
Learn to build event-driven client and server applications
I want to build a web server, a mail server, a BitTorrent client, a DNS server, or an IRC bot—clients and servers for a custom protocol in Python. And I want them to be cross-platform, RFC-compliant, testable, and deployable in a standardized fashion. What library should I use?
Use Twisted
Twisted is a “batteries included” networking engine for writing, testing, and deploying event-driven clients and servers in Python. It comes with off-the-shelf support for popular networking protocols like HTTP, IMAP, IRC, SMTP, POP3, IMAP, DNS, FTP, and more.
To see just how easy it is to write networking services using Twisted, let’s run and discuss a simple Twisted TCP echo server:
from twisted.internet import protocol, reactor
class Echo(protocol.Protocol):
def dataReceived(self, data):
self.transport.write(data)
class EchoFactory(protocol.Factory):
def buildProtocol(self, addr):
return Echo()
reactor.listenTCP(8000, EchoFactory())
reactor.run()
With Twisted installed, if we save this code to echoserver.py and run it with python echoserver.py, clients can now connect to the service on port 8000, send it data, and get back their echoed results. Read more…
Four short links: 17 April 2013
Software Archive, Self-Tracking, Provisioning, and Python Ciphers
- Computer Software Archive (Jason Scott) — The Internet Archive is the largest collection of historical software online in the world. Find me someone bigger. Through these terabytes (!) of software, the whole of the software landscape of the last 50 years is settling in. (And documentation and magazines and …). Wow.
- 7 in 10 Doctors Have a Self-Tracking Patient — the most common ways of sharing data with a doctor, according to the physicians, were writing it out by hand or giving the doctor a paper printout. (via Richard MacManus)
- opsmezzo — open-sourced provisioning tools from the Nodejitsu team. (via Nuno Job)
- Hacking Secret Ciphers with Python — teaches complete beginners how to program in the Python programming language. The book features the source code to several ciphers and hacking programs for these ciphers. The programs include the Caesar cipher, transposition cipher, simple substitution cipher, multiplicative & affine ciphers, Vigenere cipher, and hacking programs for each of these ciphers. The final chapters cover the modern RSA cipher and public key cryptography.
R as a Programming Language
Moving beyond traditional tools makes data analysis faster and more powerful
Garrett Grolemund is an O’Reilly author and teaches classes on data analysis for R Studios.
We sat down to discuss why data scientists, statisticians, and programmers alike can use the R language to make data analysis easier and more powerful.
Key points from the full video (below) interview include:
- R is a free, open-source language that has its roots in S-PLUS [Discussed at the 0:27 mark]
- What does it mean for R to be a programming language versus just a data analysis tool? [Discussed at the 1:00 mark]
- R comes with many useful data analysis methods already implemented, so you don’t have to start from scratch. [Discussed at the 4:23 mark]
- R is a mix of functional and object-oriented programming that is optimal for handling data structures that data analysts expect (e.g. vectors) [Discussed at the 6:08 mark]
- A discussion of using R in conjunction with other languages like Python, along with packages that help with this [Discussed at the 7:30 mark]
- Getting started using R isn’t really any harder than using a calculator [Discussed at the 9:28 mark]
You can view the entire interview in the following video.
Related:
Python data tools just keep getting better
A variety of tools are making data science tasks easy to do in Python
Here are a few observations inspired by conversations I had during the just concluded PyData conference1.
The Python data community is well-organized:
Besides conferences (PyData, SciPy, EuroSciPy), there is a new non-profit (NumFOCUS) dedicated to supporting scientific computing and data analytics projects. The list of supported projects are currently Python-based, but in principle NumFOCUS is an entity that can be used to support related efforts from other communities.
It’s getting easier to use the Python data stack:
There are tools that facilitate the dissemination and sharing of code and programming environments. IPython2 notebooks allow Python code and markup in the same document. Notebooks are used to record and share complex workflows and are used heavily for (conference) tutorials. As the data stack grows, one of the major pain points is getting all the packages to work properly together (version compatibility is a common issue). In particular setting up environments were all the pieces work together can be a pain. There are now a few solutions that address this issue: Anaconda and cloud-based Wakari from Continuum Analytics, and cloud computing platform PiCloud.
There are many more visualization tools to choose from:
The 2D plotting tool matplotlib is the first tool enthusiasts turn to, but as I learned at the conference, there are a number of other options available. Continuum Analytics recently introduced companion packages Bokeh and Bokeh.js that simplify the creation of static and interactive visualizations using Python. In particular Bokeh is the equivalent of ggplot (it even has an interface that mimics ggplot). With Nodebox, programmers use Python code to create sketches and interactive visualizations that are similar to those produced by Processing. Read more…
Four short links: 5 November 2012
Psychology in a Nutshell, IRS Data, Fulltime Drone CEO, and SQL Injection
- The Psychology of Everything (YouTube) — illustrating some of the most fundamental elements of human nature through case studies about compassion, racism, and sex. (via Mind Hacks)
- Reports of Exempt Organizations (Public Resource) — This service provides bulk access to 6,461,326 filings of exempt organizations to the Internal Revenue Service. Each month, we process DVDs from the IRS for Private Foundations (Type PF), Exempt Organizations (Type EO), and filings by both of those kinds of organizations detailing unrelated business income (Type T). The IRS should be making this publicly available on the Internet, but instead it has fallen to Carl Malamud to make it happen. (via BoingBoing)
- Chris Anderson Leaves for Drone Co (Venturebeat) — Editor-in-chief of Wired leaves to run his UAV/robotics company 3D Robotics.
- pysqli (GitHub) — Python SQL injection framework; it provides dedicated bricks that can be used to build advanced exploits or easily extended/improved to fit the case.
Checking in on Python
Guido Van Rossum on the state of Python and the two services that are helping to push it forward.
Guido van Rossum is the creator of Python. I recently had the opportunity to talk with him about the state of the language.
You probably don’t realize it, but Python’s capabilities are pushed every time you use YouTube and Dropbox. During our interview, Van Rossum said both of these services are at the forefront of Python’s development.
“Whenever someone clicks on a [YouTube] video, they will see HTML that was generated from Python,” he said. “That’s definitely pushing the limits.” [Discussed 27 seconds in — you can see the scalability presentation that Van Rossum mentions during this segment here.]
On the Dropbox side, Van Rossum said the service’s clients for Linux, Windows and Mac are all implemented in Python. You’re also downloading a miniature version of the Python runtime when you’re using Dropbox. [Noted at 1:20.]
Van Rossum also spoke about the lengthy transition Python has undergone from Python 2 to Python 3. “If you want improvements to your Python … now is the time to start trying out Python 3.” Why? While the changes to the language are actually quite small, with the exception of unicode handling being completely overhauled, Python 3 is a better, faster version of Python. In addition, many third parties like Django are coming on line with libraries and frameworks for Python 3. [Discussed at the 7:01 mark.]
Four short links: 27 July 2012
Weibo cf Twitter, Rendering Fonts, Clothing Manufacturing, and Profiling Python
- Social Media in China (Fast Company) — fascinating interview with Tricia Wang. We often don’t think we have a lot to learn from tech companies outside of the U.S., but Twitter should look to Weibo for inspiration for what can be done. It’s like a mashup of Tumblr, Zynga, Facebook, and Twitter. It’s very picture-based, whereas Twitter is still very text-based. In Weibo, the pictures are right under each post, so you don’t have to make an extra click to view them. And people are using this in subversive ways. Whether you’re using algorithms to search text or actual people–and China has the largest cyber police force in the world—it’s much easier to censor text than images. So people are very subversive in hiding messages in pictures. These pictures are sometimes very different than what people are texting, or will often say a lot more than the actual text itself. (via Tricia Wang)
- A Treatise on Font Rasterisation With an Emphasis on Free Software (Freddie Witherden) — far more than you ever thought you wanted to know about how fonts are rendered. (via Thomas Fuchs)
- Softwear Automation — robots to make clothes, something which is surprisingly rare. (via Andrew McAfee)
- A Guide to Analyzing Python Performance — finding speed and memory problems in your Python code. With pretty pictures! (via Ian Kallen)