How to remove files with special characters in Linux
I recently found myself having to remove a file with special characters. Using rm only gave me “rm: unrecognized option”, which forced me to take off the gloves.
To remove a file with dashes, slashes or other special characters, the easiest way is to access the file using its inode. To get the inode of a file, just do a ls -li. When you have that number, use find to delete the file using the following command:
find . -inum [inode] -exec rm -i {} \;
Good hunting!
Shell script to show a file and optionally delete it
I needed to open a number of files and depending on the content, delete them. Here is a quick bash script to cat the file and ask a question to delete the file afterwards.
#!/bin/bash
VIEWER="cat"
if [ -d $1 ]
then
DIR=$1
else
echo "Enter a directory to clean"
exit 1
fi
for file in `ls $DIR`
do
$VIEWER $DIR/$file
echo -n "Delete file (y/n)? "
read RESP
if [ $RESP == "y" ]
then
echo "Deleting $DIR/$file"
rm -f $DIR/$file
fi
done
You can change the viewer to something else, if you don’t want to use cat, and instead of deleting the file, you might want to move it somewhere else instead.
Unity, and Ubuntu Light
Unity, and Ubuntu Light is a post from Mark Shuttleworth, describing the future of Ubuntu as a netbook and light web browsing operating system.
A few months ago we took on the challenge of building a version of Ubuntu for the dual-boot, instant-on market. We wanted to be surfing the web in under 10 seconds, and give people a fantastic web experience. We also wanted it to be possible to upgrade from that limited usage model to a full desktop.
He continues to talk about touch being a target input for this OS, so making it easy to navigate, changing applications with the touch of a finger has added to the design decisions.
Ubuntu 10.04 LTS released
Ubuntu 10.04 LTS was just released with amazing new features and fixes including:
- New Ubuntu Software Center
- New splash screen using plymouth
- UbuntuOne Music Store (with Rhythmbox integration)
- New “Memenu” that lets you post to twitter/identi.ca (using Gwibber) and also change your IM status in Empathy.
- Lots and lots of bug fixes
This is a LTS release, meaning that it will be supported until 2013 for desktops and 2015 for servers.
Getting push replies for Twitter for free
I wanted to get notified on Twitter replies and mentions instantly, both on my Mac and on my iPhone. Since Google released Push for the iPhone a couple of weeks ago, an email based solution would be perfect.
Get Twitter Mentions is a bash script well suited for this task. It is easily customizable and you can run is as often as you like. Just add a crontab like the following on a Linux server of your choice:
*/2 * * * * /home/joch/bin/send-mentions-as-email.sh
This will execute the script every two minutes and email you if there are any new updates. Just make sure that you keep the requests well within the API limits (currently 150 requests per hour), or your API access may be revoked for a while. The emails are very well designed, having the tweets in the subjects for easy viewing and a full profile and other types of information right there in the email.
The following pictures are taken from the author’s site, but shows in a very clear way how things look.
Safe document writing using Dropbox
Lots of people who write articles or create content in any form, often find themselves generating lots of files. A writer will for instance probably have lots of article drafts laying around. Everyone have different solutions for revision control and backup, ranging from a simple manual file copy to using a full-fledged revision control system such as Subversion.
For everyone else, there is a simple solution for keeping backups of your work in progress, as well as being able to retrieve any previous revision. In addition to all this, it even lets you sync files between multiple computers and access your files online from any computer with internet access.
I guess you know by now that I am talking about Dropbox, a service available for Windows, Mac and Linux. It installs a small application on your computer which monitors a configurable directory for changes and uploads them automatically to the Dropbox servers.
The free version offers 2 GB of space, which should be enough for most people. For photographers and other people dealing with lots of large files, there also a premium option available which gives you 50 GB for $99 per year.
The web interface is beautiful and easy to use for navigating your Dropbox and downloading the files. This is also the place for viewing older revisions for your files and delete, copy, rename and delete them.
A very handy feature is the ability to share folders with other Dropbox users! If you are working together with other people in a project, just share a folder between you and everyone will instantly have access to all changes in the project folder – automatically.
There is even a way of sharing files with non Dropbox users. There is a special folder in the root of the Dropbox named “Public”. Putting files here makes it possible to right-click on the files and copy a public URL for it. To let other people download the file, it’s just a matter of sharing the link with them. They can’t of course make changes to it, nor view its revision history.
Another special folder in the Dropbox root is the Photos folder, which creates instant photo albums for viewing on the web by anyone. This is definitely the easiest way of getting a photo album up on the web, since you only need to copy or move the pictures to this special folder on your computer – Dropbox does the rest.
All iPhone users out there, and possible other phone owners, can access the iPhone web interface too for downloading files in the Dropbox. It is even possible to view the uploaded photo galleries.
There is a tour available on the website which explains all features more in-depth.
Upcoming features include:
- Timeline based undo
- Online visualization for any file type
- An iPhone application/interface that let us download files of interest (pdf, docs, pictures..)
- Watch any folder support (configurable per host)
- Better shared folder controls (permissions, etc.)
- Online edition for text files
- Add friends
- Improve Upload Speed
- Group accounts
If you decide to give Dropbox a go, consider using my referral link when you sign up. That way, both you and I get additional storage for free!
Disclaimer: From this article it may seem like I work for Dropbox, but I don’t. I just like their service a lot!
Spotify — the future of music
Spotify, now in public beta, takes a whole new approach to music. All music resides on the Spotify servers, and as a user, you stream the music using the internet.
Where it gets interesting is that you can either pay a fixed fee and listen all music you want without any interruptions. There is also a free option, where short commercial messages are inserted between songs in intervals which seem to be around half an hour.
Since the software is in public beta, you need an invite to use the service. If I happen to have an invite, I will of course share it with commenters to this post. There is another option though — paying for the service for a month. If you like the service after using it for a month, just continue paying or terminate the subscription. Your account will then become a free account, and you can continue to use Spotify, but with the ads injected.
There is a Windows and Mac client available at this moment. There have been reports that the Windows version works fine in Linux using Wine, so definitely try it out.
So how does is look like? The following view shows what is new in Spotify.
The next image shows the top list. This can be customised to only show a specific country or the whole world.
The main album view looks like the following. On the top, there are five top hits for that particular artist. It is followed by all albums available, and singles follow. The next section is devoted to albums where the current artist has one or more tracks, which are highlighted.
I have some suggestions to make Spotify even better (in no particular order):
- Last.FM integration.
- Music browser (browse by genre, year, artists etc.).
- Mechanism to correct faulty tags.
- Social part where users can share songs, playlists and watch each others listening habits.
- iPhone application
Some of these points seem to be in progress right now, but time will have to tell what will happen in the future when the services goes mainstream, which I am sure they will, as long as they can provide a smooth service without interruptions and errors.
Updating the Postfix HOWTO
The time has come to update the Postfix HOWTO for Debian and Ubuntu on the website. There are lots of changes done in the Postfix and Dovecot distribution, as well as input from lots of commenters. The release of the latest Debian distribution called Etch, there have been some major improvements of various components.
Within the next couple of weeks, I will publish an updated version with hopefully additional features such as DSPAM and other important components for creating an enterprise grade mail server.
I have realized that it is impossible to manage such a large document as a normal post, so if there is time, I will create it using Latex or some other tool to manage large documents.
Automate system administration tasks using Nagios
As a system administrator, one often have to do repetitive tasks such as checking for free disk space, check mail queues and monitor critical services. If there are only a handful of servers, this task may not be very intimidating, but there are many times when there are many servers to monitor, or just for the sake of automation. This is where Nagios comes in.
Nagios is a host and service monitor designed to inform you of network problems before your clients, end-users or managers do.
This is exactly what we need to make an automated system for monitoring! I will not go into details on how to set this up, since there is an excellent quick start guide available on the website. Instead I will focus on how Nagios has eased the burden of managing a large number of servers.
I have ready made templates for servers and when a new server is added, I just create a copy of the template and add or remove the services needed to monitor the server.
Public services are easy to monitor directly from Nagios, but private data such as disk space and CPU load demands a local service running on each of the servers. This is where NRPE comes into play. NRPE is a daemon which listens on the network and will respond to Nagios queries, using standard Nagios plugins. In Debian and Ubuntu, just install the nagios-nrpe-server package, and in Windows NSClient is very usable and easy to configure.
The last thing is alerts management. All servers that someone else manages, or is in charge of, should receive the Nagios alerts for that server. It will dramatically lighten the administration burden if it is possible to delegate as much as the server / service responsibility to other people. For extremely critical services, there should be an SMS gateway, which sends a message to the administrator or someone in charge of the server. This ensures that attention is immediately brought to the problem.
Create a fixed size network storage for Time Machine
Time Machine is a backup program built into Mac OS 10.5, Leopard. It saves all files on the computer on a USB or network drive, which can be used for restoration of individual files or the whole computer.
The normal behavior of Time Machine is to keep
- hourly backups for the past 24 hours
- daily backups for the past month
- weekly backups until your backup disk is full
It is the last point that might cause some trouble for some people, since many people might share the drive with other type of data. There has to be some way to limit the size of the backup volume. This is my approach.
Preparing an image
The first step is to create an image to hold the backup filesystem. If you want this filesystem encrypted, have a look at Mounting encrypted volumes, otherwise just follow the following steps. The image will be created as /ext/timemeachine.img and it will be mounted in /ext/timemachine.mnt.
dd if=/dev/zero of=/ext/timemachine.bin bs=1G seek=250 count=1losetup /dev/loop1 /ext/timemachine.binmkfs.ext3 /dev/loop1tune2fs -c0 -i0 /dev/loop1losetup -d /dev/loop1mkdir /ext/timemachine.mnt
The first thing is to create an image file, and using the dd command we create an empty 250GB file, which will contain the backups. The next step is to setup the image as a loop device, which makes it possible to mount it as usual. loop1 is currently used, but if you know that it is occupied, feel free to choose another device.
The next step is to edit /etc/fstab and add a line which will automatically mount the filesystem when the computer boots.
/ext/timemachine.bin /ext/timemachine.mnt ext3 loop=/dev/loop1 0 0
Then we will mount all filesystems and verify that it has indeed been mounted.
df -h/ext/timemachine.bin 248G 188M 235G 1% /ext/timemachine.mnt
There should be a line like the above if everything is working correctly. The last step is to set the correct permissions for the directory for your user.
chown -R joch /ext/timemachine.mnt/
Setting up the Samba share
To connect to the server, it is necessary to setup the Samba server. Create a share like the following in /etc/samba/smb.conf.
[tmbup]comment = Time machine backupspath = /ext/timemachine.mntbrowseable = yesread only = Noinherit permissions = noguest ok = noprintable = no
Now just reload Samba and add a user if you have not done so before.
invoke-rc.d samba reloadsmbpasswd -a joch
Setting up Time Machine
Connect to the share in Finder as usual.
Open up the Time Machine preferences and click Change Disk. It should give you a dialog like this, and Time Machine should then be enabled.
If you get the error “Time Machine Error: The backup disk image could not be created.”, you will need to do some magic on the server.
You need to start the backup once again, but this time you will have to be quick and copy the directory it creates on the server. Once Time Machine has finished, the original directory will be deleted, so just copy the saved directory back to the same place.
Now run the backup again, and it should complete successfully!
This behaviour is very strange, but the above trick always solves the problem.