Tag Archive for 'debian'
A lot of developeres I knew back in college loved using Macs for their development machines, and you don’t have to look very far to see this opinion reflected on the Internet. In total, I’ve been using a Mac as a development machine for about five months. And now that I have a spare moment to write, I want to share my experience coming from using Debian Linux as a development machine to coding Java on a Mac.
The Environment
OSX as an environment is just different initially. Using the Apple key instead of the control key is odd to get used to, so there was some loss of productivity (and a lot of frustration) when my muscle-memory keyboard shortcuts weren’t working. I got over the change after a couple weeks, but it still is a pain. The placement of the Apple key is two keys further in than control, and I now have to move my hand a decent amount to hit my very frequently keyboard shortcuts.
The windowing system also takes some getting used to. Close and application and it doesn’t actually close? What gives? I haven’t found a single instance where that has been a useful feature to have. It just adds a separate action to closing a process.
On those lines, windows can only be resized from the bottom right corner. If you want to expand something by grabbing an edge, or making something bigger by dragging an upper edge, you can’t. After growing up on Windows and then Debian with KDE, the instinctual grab and drag resulting in a non-action is annoying. The borders are there, but they aren’t functionally doing anything other than separating the window from other windows.
OSX is also missing virtual desktops. From a developer’s standpoint, all I can do is issue a giant WTF. Having separate desktops allows for the separation of distracting tasks and productive tasks. Having e-mail, instant messenger, and iTunes sharing a space with Eclipse, documentation, and my xterms (more on that in a sec) is extremely counter-productive. The argument can always be made that only applications you are using should be open, but switching workflows should be as simple as hitting a shortcut – not by manually managing windows.
A very minor point is that OSX does not feel snappy. My machine is a couple years old, so that could have something to do with it – but having a few xterms open and eclipse in the background should not cause a noticeable delay when switching to and typing into a new text field.
The Tools
The biggest thing that initially irked me about OSX was its lack of several very common tools. For example, OSX offers curl as a web retrieval tool instead of wget, claiming they are functionally equivalent. They absolutely are not. Wget has a mirroring feature that I found myself sorely missing when trying to pull down a multi-leveled source tree without version control from a web page.
Also missing isn’t as much of a tool, but a feature that is completely missing: the /proc filesystem. On a Linux box, a /proc filesystem can be easily enabled (if it isn’t by default) which contains information about the kernel, processes, devices, the processor, and a HUGE amount of infromation that comes in pretty handy in unexpected ways while developing. Not having this resource available causes headaches at inopportune times.
Other tools are available, but modified. Tools such as netstat show different information and have slightly modified options than the Debian counterparts. The result: figuring out how to display which process was tied to a port took nearly 15 minutes to figure out instead of taking a couple seconds to issue a command. Another example is top: processes can’t be listed as trees in the OSX version of the tool. It is small subtleties such as this that interrupt me when I am deep in a mental debugging stack – subtleties that kill productivity and add frustration.
A smaller issue was the lack of X11 and xterm. Console is fine for small tasks, but lacks the power and customizability of an xterm. Plus, I don’t want to reconfigure a console to match my xterm. I just want to pull over my .Xdefaults and have things the way I had them before. Luckily, Apple provides X11 in their developer pack, so installation wasn’t an issue.
Installation and Package Management
Application installation on a Mac is incredibly easy for user applications. Things like Omnigraffle (my very favorite diagramming software EVER) install with a drag and drop from a disk image to the applications folder.
However, installing common Unix tools (like wget, for reasons mentioned above) was a problem completely ignored by Apple. There is no package manager that comes with OSX by default for these kind of tools, and compilation is left up to the user. As a developer, this matters to me. In Debian, the power of the APT package management system made installing a new tool a single command process. Apple very easily loses here in terms of ease of use.
Libraries are also managed in a way that was initially odd. OSX considers applications like Java to be a framework, so access to something like JAVA_HOME is hidden behind a symlink. Not being in a standard location was initially annoying (/usr/lib or something similar), but once found, I was already a huge fan of the system. Keeping a symlink to a library is a Good Thing ™, because upgrades can be made seamlessly to new versions without requiring any updates to the shortcuts that point to the library. However, it could be done better. Debian uses an ‘alternatives’ system for all major libraries. Now multiple versions of any library can be set as the system-wide default at any time without requiring shortcut updates by allowing the alternatives system to manage the symlinks for you. This actually becomes useful in java development for allowing your application to be easily tested using different virtual machine implementations.
Conclusion
The conclusion shouldn’t come as any surprise to anyone: I hate developing on a Mac. I do no Mac-specific development, so the ease of writing Cocoa applications with Xcode means nothing to me. The features that I have come to rely on are either missing or implemented differently on OSX, leaving me with hurdles and interruptions where I absolutely do not need them.
When I am developing, I want things to just work – and I mean that from a developer’s perspective. My user experience matters less to me than having my platform enable my development productvity, and in this department OSX fails. I eagerly await the time when I can switch back to a Debian Linux workstation for my professional development, with fluid workflows and familiar tools to fuel my debugging speed and code output.
GCJ, the GNU compiler with java extensions, does a great job at compiling Java into bytecode, but still has some bugs in its libraries when dealing with Swing components. Installing Sun’s Java packages on Debian thus is occasionally necessary, and has historically been a chore. I won’t list the process here to even show my distaste for it – it just wasn’t very fun.
Things are much easier now, though. Just make sure a non-free package repository is listed in your sources.list, and things become magic:
sudo echo "deb http://ftp.us.debian.org/debian/ lenny non-free" >> /etc/apt/sources.list
(Note that if you aren’t using lenny, you should change that. Also, feel free to choose a different mirror.)
Now update your package repository:
sudo apt-get update
And finally install whichever Sun Java packages you want!
sudo apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin
Cheers to Matthias Klose (Ubuntu), Juergen Kreileder (Blackdown), Barry Hawkins, Jeroen van Wolffelaar, and the other folks behind debian-java for adding these packages to Debian’s repositories. It is another push that greatly enhances the usability of the project for both developers and users alike.
As is plastered all over the Internet, Ubuntu 8.04 Hardy Heron was recently released for the public to feast upon. I really don’t care about the release itself, but I *do* care about Mark Shuttleworth’s blog post regarding the release. Specifically:
We all owe a great deal to the team who make Debian’s “unstable” repository possible, and of course to the upstream projects from GNOME and KDE through to the Linux kernel.
Read the rest of it here.
I think Mark’s statement shows maturity in the Ubuntu project, and I respect him for showing the open source community some love.
I’m not affiliated with the Debian project in any official capacity (yet!), but I’m happy they are getting the credit they deserve. Kudos to the Debian team and every package maintainer – you don’t hear it enough, but there is a large percentage of the technically aware population that appreciates the work you do beyond measure.
I got hit with a weird bug when I upgraded to VMWare 6.0.3. Before, my Debian Linux guest could happily browse the Internet with a NAT interface. After, I could resolve both internal and external IP addresses, but could not resolve any domain names using DNS.
I can only guess at why, but I believe the problem was not the configuration of my guest, but rather that the problem lays in how VMWare deals with the DNS queries. I use DHCP to obtain an address from the built-in DHCP server, and I was being assigned a DNS server along with my IP (as verified by checking /etc/resolv.conf), but it wouldn’t resolve any names for me.
My solution for now is to step around the assigned DNS address given by the DHCP server, and choose another one instead. I opted to use OpenDNS’s servers, and configuring Debian to use them is trivial – just open /etc/resolv.conf as root with your favorite editor, and change the ‘nameserver’ line to use OpenDNS’s DNS servers. Mine looks like this:
domain localdomain
search localdomain
nameserver 208.67.222.222
Restarting the network isn’t even necessary, the next named request you make should be resolved successfully and everything should be peachy!
If you want to make the change more permanent, you can assign a static IP and a static nameserver to the active interface of your guest operating system (hint, /etc/network/interfaces), which will circumvent the DHCP server. If you want to keep using the DHCP server, you’ll have to change the nameserver line every time you request a new address.
Hope this helps someone else!
This past academic quarter I elected to do an independent study. A lot of hype has been surrounding web development over the past few years, and the python scripting language for even longer than that. Figuring it was time to immerse myself in both subjects and the technologies related to them, the independent study turned into a full four credit project aimed toward creating a web application for administering quizzes. Because I only had ten weeks to learn everything involved with developing a web application on top of the language, I coerced my good friend David Brenner into doing it with me. Development is done, and I now feel like I can talk intelligently about my experiences.
The project was developed using straight python, utilizing the mod python Apache module to serve Python Server Page (psp) files. Python isn’t my favorite language to code with, it is lacking some things that I’ve come to expect from a scripting language, and some things are just silly. Examples of lacking features include the ability to get an iterator for a collection. Iterators that grant references both to the next and previous element of a collection grant superior control and the ability to keep state while iterating. Python only allowing you to move forward through a collection with its “for x in collection:” construct really hurts while parsing large sequences of text, and having to revert to array notation and keep an index is much less human readable than just manipulating an iterator.
Annoyances with the language include python being a scripting language that doesn’t default to printing the string value of non-string objects, and using procedural style method calls intermixed with object-oriented style. For example, to print an object (say an array), you can’t just do print(“Array contents: “ + arrayName). You have to manually return the string value by using print(“Array contents:” + str(arrayName)) in order to avoid an error that print() was expecting an object of type string. I know python’s mantra is “explicit over implicit,” but languages are tools that you learn how to use. As I learn more, I expect to gain proficiency with the tool in order to save me time in the future. Python is easy to learn and easy to use, but I have yet to find any tricks with the language that save me time or make my job easier than it would be with another language. One last thing that I simply do not understand is the necessity to include a reference to “self” in method and constructor definitions. I strongly prefer the method signature to match the required call, python’s decision to branch from that convention is one that I can’t imagine will have a justification strong enough to alter my opinion of the practice.
As it goes with web development, we also used javascript, a MySQL database backed with InnoDB table engines, LDAP for authentication, HTML, and CSS. Development was utterly boring. Being a computer science student, I enjoy working on new things that present challenges and reward creative solutions. The most thought intensive part of this project was the parser run on imported quizzes using a format we decided on – and that was developed in the fourth week. Everything else was the brainless transfer of requirements to code. Some of the people I go to school with are attracted this kind of development, and I think I know why: frontend web dev is easy. Writing the backend logic isn’t bad, but the interface code and associated handlers could be composed by a high school student with some spare time and only rudimentary knowledge. The things learned in computer science definitely aren’t applied in that domain.
Finally, to wrap up loose ends: object relational database design is neat and very handy, javascript is odd at first but easy to pick up, dealing with web requests and forms is tedious, debugging web applications is a *bitch,* and Debian Linux with Apache, Subversion, screen, and vim makes for a great development and hosting environment.
Web development has its place, but it isn’t for me. I’m capable with python, but I’ll stick with ruby. And if you introduce yourself to me, try to be more descriptive than “I’m a web developer” so my experiences don’t tarnish my opinion of you – because I bet you are more skilled at writing code than that title would suggest.
I enjoy computer security. There aren’t a lot of opportunities to study it formally within computer science, so my education in this field is entirely from what I read and practice in my own time.
Most recently, I’ve been feeling the itch to write a worm. The idea is attractive because a worm can be developed modularly with reusable components. Each individual component will increase my knowledge substantially in a different area of security, making the development a measurable goal with incremental positive feedback.
However, before development could begin, I wanted to ensure that I wouldn’t end up in court for an accidental release of one of the components gone awry. I love virtual machines as a tool to aid in the development process, so the solution was immediately obvious – create a multi-host virtual network that is isolated from the world. Further, I wanted each machine on this isolated network to occasionally be able to access the Internet to retrieve updates or tools, so the isolation needed to be complete but /controllable./ The final requirements of the virtual network ended up looking like this:
- Isolated network except when explicitly given access to the Internet
- Multiple hosts with different operating systems
- Must be able to easily add and remove hosts
- All hosts on the network must both default and fail to isolation
The way to implement this using VMWare Workstation (and I’m sure other products in their virtualization line) is to utilize teams. Teams are a ‘wrapper’ of a sort that encompass multiple VMs with additional configuration. When you start a team, each virtual machine included in the team’s configuration is also started. The team can be configured to also provide a virtual network segment for the virtual machines to use, which when paired with each VM in the team being configured with ‘host only’ network access, results in a virtual isolated network.
The team doesn’t provide DHCP though, which means the network has to be maintained with static address and modifications to each machine’s host file. This hardly met my requirement for easily adding and removing hosts from the network. Creating a host that would act as the network server fulfills this requirement, and will also facilitate network control access. As we continue on, please note that I’m using Debian Linux with a 2.6.x kernel, and all of the commands I give below and edits to configuration files *must be done as a superuser.*
Enough setup: time for implementation. To speed the process, I created two base images, one Windows XP SP2 install, and one Debian Lenny netinstall with a 2.6.x kernel. Each image was updated to include the latest patches, user accounts were created, and standard tools were installed. Once these base images were created, they were set aside to never be modified. Clones of the base images are created for each of the expendable hosts, and one clone of the Debian base image was used as the only ‘permanent’ member of the team. All members of the team share one virtual network segment, and have one interface. The only exception to this is the network server VM, which is dual-homed to be connected to both the virtual network and the Internet via NAT.
All hosts default to DHCP, so cloned images have no need for additional configuration when added. The network server is the only machine that had be set up specially. The bind9 and dhcp3-server packages were obtained (for DNS and DHCP, respectively) using Debian’s awesome package manager:
apt-get install bind9 dhcp3-server
Configuring bind is trivial, it defaults to forwarding DNS requests, so nothing is required as far as configuration unless you want to. dhcpd, provided by dhcp3-server, is a little more complicated. First, the interface connected to the isolated network must be set up to have a static address in the subnet in which you will be offering IP addresses, like 10.10.10.1 for the 10.10.10.x subnet or 192.168.30.1 for the 192.168.x.x subnet. It would be wise to modify your interface configuration to make this change survive rebooting.
/etc/network/interfaces:
auto lo eth0 ethiface lo inet loopback
iface eth0 inet static
address 10.10.10.1
netmask 255.255.255.0iface eth1 inet dhcp
The external interface is eth1, and is configured with DHCP since it is NAT routed. The internal interface is eth0, and is given an ip of 10.10.10.1 with a subnet mask of 255.255.255.0. (This means that the last quartet of the IP address is variable and available for use.) Next comes the configuration for dhcpd:
/etc/dhcp3/dhcpd.conf:
default-lease-time 600;max-lease-time 7200;
authoritative;option domain-name-servers 10.10.1.1 192.168.30.1
subnet 10.10.10.0 netmask 255.255.255.0 {
range 10.10.10.2 10.10.10.254;
option routers 10.10.10.1;
option ip-forwarding off;
option broadcast address 10.10.10.255;
option subnet-mask 255.255.255.0;
}
Here we are saying that the subnet is 10.10.10.*, and that we will assign addresses from 10.10.10.2 – 10.10.10.254. The other options should be self-explanatory – read up on networking if you have questions. As it stands, when the interfaces are brought down and back up and dhcpd is started, addresses will be assigned to all virtual machines sharing that network segment. If this is all you want, just issue:
ifdown eth1 eth0
ifup eth1 eth0
/etc/init.d/dhcpd3-server start
And you are done! The machine now will serve DHCP to the isolated subnet, while maintaining separate access for itself to the Internet.
However, if you want to continue on to enable Internet access for other hosts on the isolated network, we still have some work to do.
My solution for this involves iptables and masquerading. Before we do anything, we’ll need to enable IP forwarding. This can be done in multiple ways, but the most reliable for me has been the following simple command:
echo 1 > /proc/sys/net/ipv4/ip_forward
With IP forwarding enabled, we can now utilize the masquerading features of iptables, the Linux firewall. By creating rules that will take packets coming in from our internal network’s interface and sending them out on our external interface, in addition to creating a complementing rule that will accept return packets coming in from the external interface headed for the isolated host, we can accomplish this. The individual rules for my setup are:
iptables -t nat -A POSTROUTING -s -o eth1 -j MASQUERADE
iptables -A FORWARD -d -i eth0 -j ACCEPT
Since these are annoying to have to type in each time I want to enable access for a host, I wrote a set of scripts. The first two enable and disable access for a host or multiple hosts respectively. The third script is my emergency “oh crap” failsafe, with which a simple command I can disable all isolated hosts’s access immediately followed by bringing down the network server’s interfaces for complete assurance that whatever is going on won’t get out of the virtual network. Here they are:
enableInternet.sh
#!/bin/bash
if [ $UID -ne 0 ]; then
echo
echo "Must be root to run this program."
echo
exit 1
fi
if [[ -z $* ]]; then
echo
echo " Usage: ./enableInternet.sh <ipaddress [ipaddress2...ipaddressN]>"
echo
exit 1
fi
for ip in $@; do
# Will match an address of type 10.10.1.2, which matches our subnet
# definition
check=`echo $ip | grep -E "^([[:digit:]]{2}[.]){2}[[:digit:]][.][[:digit:]]+$"`
# If it doesn't match, print a warning and skip it
if [ -z $check ]; then
echo "Improperly formatted address $ip, skipping..."
continue
fi
# Enable Internet access for the address
iptables -t nat -A POSTROUTING -s $ip -o eth1 -j MASQUERADE
iptables -A FORWARD -d $ip -i eth0 -j ACCEPT
echo "$ip's internet access enabled..."
done
echo "Done."
blockInternet.sh
#!/bin/bash
if [ $UID -ne 0 ]; then
echo
echo "Must be root to run this program."
echo
exit 1
fi
if [[ -z $* ]]; then
echo
echo " Usage: ./blockInternet.sh <ipaddress [ipaddress2...ipaddressN]>"
echo
exit 1
fi
for ip in $@; do
# Will match an address of type 10.10.1.2, which matches our subnet definition
check=`echo $ip | grep -E "^([[:digit:]]{2}[.]){2}[[:digit:]][.][[:digit:]]+$"`
# If it doesn't match, print a warning and skip it
if [ -z $check ]; then
echo "Improperly formatted address $ip, skipping..."
continue
fi
# Disable Internet access for the address
iptables -t nat -D POSTROUTING -s $ip -o eth1 -j MASQUERADE
iptables -D FORWARD -d $ip -i eth0 -j ACCEPT
echo "$ip's internet access disabled..."
done
echo "Done."
blockAll.sh
#!/bin/bash
if [ $UID -ne 0 ]; then
echo
echo "Must be root to run this program."
echo
exit 1
fi
echo "Disabling Internet access for all hosts on 10.10.1.0/255.255.255.0..."
iptables --flush
iptables --delete-chain
iptables -t nat --flush
iptables -t nat --delete-chain
ifdown eth0 eth1
echo "Done."
I alias’d all the commands in my shell’s configuration scripts and prefixed them with sudo so they may be executed quickly and from anywhere on the system. If you’ve read this far, you should too – at least for the blockAll script. You don’t want to be fumbling around trying to remember where you put the script when you need complete isolation 30 seconds ago :)
I know this post was long, but there was a lot to cover. With this setup, hosts can now be easily added thanks to DHCP, Internet access is manually granted and defaults to none, and the environment is completely homogeneous. Perfect for worm development, malware analysis, or what have you. If you replicate this environment, let me know how it works out for you and what improvements you make. I’m always interested in making better systems!
One of the relatively recent additions to KDE/Gnome has been the ‘network-manager’ tool. It attempts to make the life of its users easier by automagically managing your interfaces. I can’t comment on its effectiveness during a normal user’s session, but my experiences were frustrating.
I guess it isn’t fair to say network-manager’s operation was frustrating, but rather that it was running without my knowledge, causing weird behavior. I’m setting up dhcpd in a virtual machine to provide addresses for an isolated network. Normally this requires that you only set up the interface, grab the package, set up the subnets you want to assign addresses to in /etc/dhcp/dhcpd.conf, and start up the daemon. However, this time around, I ran into some problems.
First, I attempted to set up my primary interface to use a static IP. I edited /etc/network/interfaces to set eth0 to use a static IP, set it and its netmask, and then brought the interface back up. The output from ifconfig showed no IP assignment. What!? Thinking it was something I was doing wrong, I brought the interface back down and ignored it while I configured dhcpd.
The dhcpd.conf file isn’t difficult to set up. The subnets to service were declared, as were the address ranges to use when assigning addresses dynamically. When the configuration was done, I attempted to bring up the daemon. /etc/init.d/dhcp3-server start… and… errors. “Not configured to listen on any interfaces?” Ugh. Further digging revealed that dhcpd was looking to serve addresses on a 10.10.1.0 network that I had configured, but the only active interface was listening on a 169.0.0.0 network.
Wait – an interface is up? I took it down! I again checked the syslogs, and sure enough, something is initiating dhclient to find an IP address, and upon failing, is assigning a private IP from the 169.0.0.0 range to my interface. The root cause of it ended up being network manager trying to maintain a connection on the interface utilizing its own configuration. Stopping the network-manager service immediately fixed the problem – bringing up the eth0 interface resulted in it being assigned the static IP I wanted, dhcpd ran without a problem, and I was happy.
I have some gripes with network-manager from this experience.
First, I edited /etc/network/interfaces to make the changes to eth0’s configuration. That is the way changes have been made to interfaces since before I started using Linux. I understand what network manager was trying to do, but two things perplex me. Why did it block my static IP assignment when I tried to bring the interface up? It wasn’t even immediately overridden, there was literally no address assigned to the interface. And also, why, WHY is it ignoring standard configuration files? /etc/network/interfaces should remain the be-all is-all configuration for network interfaces. Using another configuration file just makes things confusing.
Second, I believe the behavior of network-manager needs to be changed. I don’t know how this would be possible, so it may be an unreasonable thought – but if I bring down an interface manually, I don’t want it back up until I manually bring it up. Network-manager bringing up that interface after I had manually taken it down was confusing as all hell, and I certainly don’t use Linux to have my machine work against me.
I have disabled network manager permanently on my system. For a standard user (are there any ’standard’ Linux users?) it may function well, but for anyone that wants control of their machine, I suggest they do the same.
Ubuntu seems to be the latest, greatest thing in the world of operating systems. The creators of Ubuntu have done some good things in terms of making Linux easier to use and in providing guidance for Linux newbs. But with newbs you bring ignorance, and there are a couple things I’ve been seeing that are kind of ridiculous.
First, I saw a post stating that “I’m of the opinion that Ubuntu could not exist without Debian.” The poster of that comment is Mark Shuttleworth, the founder of the Ubuntu project. Just as you would expect Bill Gates to downplay the significance of taking ideas from other operating systems, Mark is downplaying his project’s dependence on Debian. The quote above should be “Ubuntu absolutely could not exist without Debian.” That is it – no opinions, just solid fact. Ubuntu is being developed by a relatively small team, and what they achieve seems impossible for their size. Their secret is simple: work on the Debian project is being done by coders working to advance the project, not for money, and Ubuntu is a thin interface on top of that work. I’m not bashing any of the paid Ubuntu developers, but the development of Ubuntu is extremely dependent on the advancement of Debian. In fact, this point has been the source of some unfavorable feelings toward Ubuntu by Debian developers, with the criticism that Ubuntu is taking a *lot* from Debian, and returning very little. So if you take one thing away from this paragraph, let it be that Debian can exist without Ubuntu. Ubuntu cannot exist without Debian.
And second, the number of Ubuntu users who believe “Ubuntu = Linux” or the same users who have never heard of Debian is surprisingly high. If one of you happens to be reading this post, here is a simple analogy to explain the truth. The Linux kernel is the component common to all distributions of Linux – you could say that it is Mr. Potato Head’s body. Distributions of Linux are different sets of packages on top of the kernel that enable the user to do different things easily. You could call these the accessories you put on Mr. Potato Head. Debian and Ubuntu are such distributions. However, Ubuntu is built on /top/ of Debian, so you could call Ubuntu the paint on the accessories on Mr. Potato Head. All in all, there was extra work to make those accessories more appealing, but that work is trivial when compared to what it took to make the accessories themselves. You can make a Debian installation behave exactly like an Ubuntu installation with very little work (just by grabbing the packages, possibly from a different package level), but doing the opposite is near impossible. To summarize: Ubuntu is Debian. Debian is not Ubuntu. Both are Linux distributions.
I really hope that this sheds some light on the dependency that Ubuntu has on Debian. And really, I would *love* to see some activism on the Ubuntu community’s side to give more back to the Debian project. The farther Debian goes, the better Ubuntu gets – they only stand to benefit.
