Gentoo Universe News

How Can Any Company Ever Trust Microsoft Again?

KWin in KDE SC 4.11 will include experimental support for Wayland, as you can read in the official 4.11 Beta 1 announcement:

KWin and Path to Wayland—Intial experimental support for Wayland was added to KWin. KWin also got many OpenGL improvements including support being added for creating an OpenGL 3.1 core context and robustness from using the new functionality provided by the GL_ARB_robustness extension. Numerous KWin optimizations are aimed at reducing CPU and memory overhead in the OpenGL backend. Some desktop effects have been re-written in JavaScript to ease maintenance.

As the Beta 1 is already available in the gentoo kde overlay you may ask what the current state is. You are right it is time to talk about it. I will try to serve you the facts.

Current state

Wayland is already packaged in portage (dev-libs/wayland), thanks to the x11 herd.

johu@elia ~ $ eix -s wayland [I] dev-libs/wayland Available versions: (~)0.95.0 (~)1.0.6 (~)1.1.0 {doc static-libs} Installed versions: 1.1.0(21:54:08 05.06.2013)(-doc -static-libs) Homepage: http://wayland.freedesktop.org/ Description: Wayland protocol libraries

KWin 4.11 Beta 1 + KWin master (4.8.10 + 9999 ) has already a build option (USE flag) for wayland.

johu@elia ~ $ eix -s kwin [I] kde-base/kwin Available versions: (4) 4.10.3 (~)4.10.4 **4.10.49.9999^m[1] [M](~)4.10.80^m[1] (**)9999^m[1] {aqua debug gles opengl wayland} Installed versions: 9999(4)^m[1](22:14:15 18.06.2013)(gles opengl wayland -aqua -debug) Homepage: http://www.kde.org/ Description: KDE window manager

It builds and links already successfully against it.

johu@elia ~ $ scan kwin dev-libs/wayland-1.1.0 dev-qt/qtcore-4.8.4-r5 dev-qt/qtdbus-4.8.4 dev-qt/qtdeclarative-4.8.4 dev-qt/qtgui-4.8.4-r1 dev-qt/qtscript-4.8.4 kde-base/kactivities-9999 kde-base/kdelibs-9999 kde-base/kwin-9999 kde-base/libkworkspace-9999 kde-base/liboxygenstyle-9999 media-libs/mesa-9.1.3 sys-devel/gcc-4.7.3 sys-libs/glibc-2.17 x11-libs/libICE-1.0.8-r1 x11-libs/libSM-1.2.1-r1 x11-libs/libX11-1.6.0 x11-libs/libxcb-1.9.1 x11-libs/libXcursor-1.1.13-r1 x11-libs/libXdamage-1.1.4-r1 x11-libs/libXext-1.3.2 x11-libs/libXrandr-1.4.1 x11-libs/libXxf86vm-1.1.3 x11-libs/xcb-util-image-0.3.9 x11-libs/xcb-util-keysyms-0.3.9 x11-libs/xcb-util-wm-0.3.9

The USE flag is globally masked for stable systems. As a side note, i realy like the stable use mask feature in EAPI 5.

Next steps

1) We need to package the Wayland compositor aka Weston in portage tree, to start a full blown Wayland session.  This task is already in progress (bug #445736), an ebuild is available in the gentoo x11 overlay. Will be finished soon hopefully.

2) Add a wayland build option (USE flag) for the KDE start script (kde-base/startkde). The USE flag will allow us to ship a modified version of it, so that we can tell KWin to use weston/wayland when starting.

So I am realy sure you will be able to play around with Wayland in starting August when KDE SC 4.11.0 is released and hit the portage tree by simply enabling a USE flag. Are you excited? You should!

Have fun!

I’d like to ask gentoo-haskell community for help. We have a nice wiki and our project page have moved there. But it seems that we don’t have enough documentation quality for end-user application. As a developers we support proper builds and tests for that packages but we are not expert users for many of them. So I’d like to ask community to add some docs and tips for applications you use. This basically means installation, advanced config (examples), interesting use cases, links to external resources (blog posts/documentation) and so on. It can help a lot for new Gentoo users.

The most interesting projects are:

• pandoc
• git-annex
• gitit
• yi
• xmonad

Thanks!

Okay the title is a mouthful for sure, but this new book from Packt Publishing is an interesting read for those who happen to use Mercurial only from time to time and tends to forget most of the commands and workflows, especially when they differ quite a bit from the Git ones.

While I might disagree with using some very unsafe examples (changing the owner of /etc/apache to your user to experiment on it? Really?), the book is a very quick read and I feel like for the price it’s sold by Packt (don’t get distracted by the cover above, that links to Amazon) it’s worth a read, and keeping it on one’s shelf or preferred ebook reader device.

Well, not sure if I can add more to this, I know it sounds like filler, but the book is short enough that trying to get into more details about the various recipes it proposes would probably repeat it whole. As I said, in general, if you have to work with Mercurial for whatever reason, go for it!

A few months ago I had the chance to get to know about one of the upcoming HP innovation in the server architecture : the Moonshot project. Now that it is public, I thought I’d take some time to talk about it as I’m convinced this is something big enough to change the way we see datacenter infrastructures and servers in general. I’ll do my best to keep it short and understandable so if you want deeper technical insights, fell free to ask or search around.

The nowdays servers

As a reminder, that’s what a standard server looks like today :

 

 

 

 

We call them pizza boxes for their flat and tasty aspect.

Then in datacenters we put them in enclosures we call racks which look like this :

 

 

 

 

 

 

Now the basic stuff to understand and keep in mind is that those racks can typically hold 42 standard servers like the one above. Datacenters are just big hangars where you store and cool hundreds/thousands of those racks.

The Moonshot project

The more processing power you need, the more racks you need, the more datacenters you need. Think of facebook or google and their enormous amount of servers/racks/datacenters around the world. Every new device (PC/tablet/smartphone) activated is a new client to an always-growing infrastructure of those powerhouses.

Basically, there’s a limit in the number of full datacenters you can build and operate eventually (not to mention powering them up) but there’s worse : the new devices/clients growth is higher than our datacenter building/powering capabilites.

The Moonshot project is one of HP’s response to this challenge : permit businesses to accommodate and serve this rapidly growing demand of devices/clients without the datacenter model collapsing. Their method ? Invent a new server architecture from scratch.

The cartridges are back !

No your Master System is still out of date… But HP’s approach to getting more servers in less space while consuming way less power resides in turning the pizza-box above into a cartridge which looks like this :

 

 

 

 

 

No black magic involved : you can now store 45 servers in 4,3 units of space. Based on their calculation, if you want the same computing power as you would have with standard pizza-boxes you’d need only one full rack of those new servers versus 4 to 6 racks filled with standard ones (depending on their config). Overall gain factors are huge :

• space divided by 4 to 6
• energy divided by 6 to 2
• cabling divided by 26 to 18
• not to mention the time saved by technicians to put everything up

That’s what the beast looks like :

 

 

 

 

 

 

Of course, they have integrated redundant switches and all the flavors of modern enclosures.

The right cartridge at the right place

Over the year, HP will launch a series of cartridges with their own specifications (RAM/HDD/CPU) which should be used to meet specific needs and designs. They can also accommodate your needs by designing your own server-cartridges if you’re in a hurry of course.

As a DevOps, I love this idea of the hardware being designed and used upon your software’s architecture because that’s closer to real efficiency and will lead both developers and IT architects to distributed and massively scaling designs.

I’ll conclude with the last thing you need to understand about this technology : it does not fit everyone’s need. Moonshot will not take over the world and replace every server around, instead it should be used as a hardware matching a real software design.

Last week we had our monthly Gentoo KDE team meeting; here are few details that are probably worth sharing.

• So far we've provided the useflag "semantic-desktop" which in particular controls the nepomuk functionality. Some components of KDE require this functionality unconditionally, and if you try to build without it, bugs and build failures may occur. In addition, by now it is easily and reliably possible to disable e.g. the file indexer at runtime. So, we've decided that starting with KDE 4.11 we will remove the useflag and hard-enable the functionality and the required dependencies in the ebuilds. The changes are being done already in the KDE overlay in the live ebuilds (which build upstream git master and form the templates for the upcoming 4.11 releases).
• After recent experiences the plan to drop kdepim-4.4 is off the table again. We will keep it in the portage tree as alternative version and try to support it until it finally breaks.
• In the meantime we (well, mainly Chris Reffett) have started in the KDE overlay to package Plasma Active, the tablet Plasma workspace environment. Since Gentoo ARM support is already excellent, this may become a highly valuable addition. Unfortunately, it's not really ready yet for the main tree and general use, but packaging work will continue in the overlay- what we need most is testing and bug reporting!

Independent of the meeting, a stabilization request has already been filed for KDE 4.10.3;  thanks to the work of the kde stable testers, we can keep everyone uptodate. And as a final note, my laptop is back to kmail1... Cheers!

Edit, 13/6/2013: Johu has posted a blog entry on how to disable the semantic desktop functionality at runtime.

Today we bumped  KDE SC 4.11 beta 1 (4.10.80) in the gentoo kde overlay. The semantic-desktop use flag is dropped in >=kde-base/4.10.80, as you may already noticed or read in dilfridges blog post. So if your hardware is not powerful enough or you just don’t want to use the feature you can easily disable it at runtime.

1. Go to the “System Settings” and search for “Desktop Search”

 

2. Uncheck at least the file and email indexer. You can also disable the “Nepomuk Semantic Desktop”.

Have fun!

 

Update

If you want to disable Akonadi you can check out this blog post.

 

GF670W

Okay I know that most of you do not follow my blog with the intention of reading about videogames, but given my Open Source time lately has been limited by me being quite busy with settling down and caring for an apartment, some updates are better than nothing. And since one of the first things that I bought for my apartment was a TV and a (new) PlayStation 3, I got to spend some time with Metal Gear Rising I thought it might be a good idea to write something about it.

First of all I have to apologize to the fans of the whole saga. I only played Metal Gear Solid 4 before, and I didn’t even finish it (my first PS3 died while I was playing it, and I had no backup of the save games — and since this happened quite a bit within the game I didn’t want to play it back from the start afterwards, I might do so now, honestly). I’m also not a big fan of stealth games (I never even completed the demo of Thief, for instance). But I liked MGS4 and I wanted to give a try to MGR simply because I loved the character of Raiden (I like blades, what can I say).

So the gameplay is nice. I love being able to cut almost everything down to pieces, especially when I’m pissed off by the neighbour’s alarm ringing at ten in the night. Or seven in the morning on a bank holiday. I admit I played through in easy mode (I wanted to vent off the stress, not cause more), and thus that might have helped with being able to get away with a basically random kind of attacks. But I liked it, and I liked the fact that it’s not entirely random. I think it might be worth a re-play now that I understand the attacks better (I’m hoping in a new game plus kind of deal).

Graphic is .. well, it’s not like there are any more games that have a bad graphics, but it could be better. It does not feel at the level of Metal Gear Solid 4. It’s also running in 720p, which is surprising for a new game. Although it might have something to do with the fact that the PS3 lacks the memory to run this properly. Oh well, not surprised I’d say, but a bit disappointed.

The soundtrack, oh wow the soundtrack! I’ve loved the soundtrack to the point I had to get it on iTunes. It charges me the same way as DMC4 did.

Unfortunately, maybe because I played in Easy mode, the game is quite too short. Yes there are downloadable chapters, and side “VR” missions, but the former you have to pay for extra, which is just a lowly trick for the publisher, and the latter is not part of the story. One “file” (chapter) consists of … one cut scene and a single batter. That’s not really that nice, in my opinion. As I said I’m going to re-play it with a bit more clue about the attacks, it’s likely going to be more enjoyable. But seriously even in “easy”, two weeks playing on and off were enough to get to the final Metal Gear… I’m not really excited about it.

I’m leaving my home base in Australia, Skydive Maitland, and venturing off. It all started with an idea, and this:

I took the seats out of the van, put a bed in the back and now I’m ready to go.

My grand plan is to go all the way around Australia. Some people tell me it is 25,000km or so. I have no real time commitments (as always), so for now I’m heading “north” – to where it is warmer. I think I’m the only guy chasing mild winter on my rtw trip, no more! I’m leaving Maitland with new friends to visit again and I did about 250+ jumps in the 3 months I was working there, good times.

Visualization of data is often needed in order to understand what the data means. When data needs to be visualized automatically, I often use the graphviz tools. Not that they are extremely pretty, but it works very well and is made to be automated.

Let me give a few examples of when visualization helps…

In SELinux, there is the notion of domain transitions: security contexts that can transition to another security context (and thus change the permissions that the application/process has). Knowing where domains can transition to (and how) as well as how domains can be transitioned to (so input/output, if you may) is an important aspect to validate the security of a system. The information can be obtained from tools such as sesearch, but even on a small system you easily find hundreds of transitions that can occur. Visualizing the transitions in a graph (using dot or neato) shows how a starting point can move (or cannot move – equally important to know ;-) to another domain. So a simple sesearch with a few awk statements in the middle and a dot at the end produces a nice graph in PNG format to analyze further.

A second visualization is about dependencies. Be it package dependencies or library dependencies, or even architectural dependencies (in IT architecturing, abstraction of assets and such also provides a dependency-like structure), with the Graphviz tools the generation of dependency graphs can be done automatically. At work, I sometimes use a simple home-brew web-based API to generate the data (similar to Ashitani’s Ajax/Graphviz) since the workstations don’t allow installation of your own software – and they’re windows.

Another purpose I use graphviz for is to quickly visualize processes during the design. Of course, this can be done using Visio or Draw.io easily as well, but these have the disadvantage that you already require some idea on how the process will evolve. With the dot language, I can just start writing processes in a simple way, combining steps into clusters (or in scheduling terms: streams or applications ;-) and let Graphviz visualize it for me. When the process is almost finished, I can either copy the result in Draw.io to generate a nicer drawing or use the Graphviz result (especially when the purpose was just rapid prototyping).

And sometimes it is just fun to generate graphs based on data. For instance, I can take the IRC logs of #gentoo or #gentoo-hardened to generate graphs showing interactions between people (who speaks to who and how frequently) or to find out the strength of topics (get the keywords and generate communication graphs based on those keywords).

When you think of SSD manufacturers, it might be obvious to think of them as Linux friendly, given they target power users, and Linux users are for the most part are power users. Seems like this is not that true for Crucial. My main personal laptop has, since last year, a 64GB Crucial M4 SSD – given I’ve not been using a desktop computer for a while it does start to feel small, but that’s a different point – which underwent a couple of firmware update since then. In particular, there is a new release 070H that is supposed to fix some nasty power saving issues.

Crucial only provide firmware update utilities for Windows 7 and 8 (two different versions of them), and then they have an utility for “Windows and Mac” — the latter is actually a ZIP file that contains an ISO file… well, I don’t have a CD to burn with me, so my first option was to run the Windows 7 file from my Windows 7 install, which resides on the external SATA harddrive. No luck with that, from what I read on the forums what the upgrader does is simply setting up an EFI application to boot, and then reboot. Unfortunately in my case there are two EFI partitions, because of the two bootable drives, and that most likely messes up with the upgrader.

Okay strike one, let’s look at the ISO file. The ISO is very simple and very small.. it basically is just an ISOLINUX tree that uses memdisk to launch a 2.88MB image file (2.88MB is a semi-standard floppy disk size, which never really went popular for real disks, but has been leveraged by most virtual floppy disk images in bootable CD-Roms for the expanded size). Okay what’s in the image file then? Nothing surprising, it’s a FreeDOS image, with Crucial’s own utility and the firmware.

So if you remember, I had some experience with trying to update BIOS through FreeDOS images, and I have my trusty USB stick with FreeDOS arriving on Tuesday with most of my personal effects that were in Italy waiting for me to find a final place to move my stuff on. But I wanted to see if I could try to boot the image file without needing the FreeDOS stick, so I checked. Grub2 does not include a direct way to load image — the reference to memdisk in their manual refers to the situation where you’re loading a standalone or rescue Grub image, nothing to do with what we care about.

The most obvious way to run the image is through SYSLINUX’s memdisk loader. Debian even has a package that allows you to just drop the images in /boot/images and adds them to the grub menu. Quick and easy, no? Well, no. The problem is that memdisk needs to be loaded with linux16 — and, well, Grub 2 does not support it if you’re booting via EFI, like I am.

I guess I’ll wait until Tuesday night, when my BIOS disk will be here, and I’ll just use it to update the SSD. It should solve the issue once and for all.

Just a quick update on Freeverb3 in Gentoo: media-libs/freeverb3-3.0.0 is now unmasked and working with media-sound/audacious-3.3.4, the latest Audacious in Gentoo. Give it a try

PS: The theme on the screenshot is Radience known from Ubuntu or x11-themes/light-themes in Gentoo.

Of course, working with a Linux desktop eventually requires you to work with an office suite. Although I have used alternatives like AbiWord and Calligra in the past, and although I do think that Google Docs might eventually become powerful enough to use instead, I’m currently using LibreOffice.

The use of LibreOffice for Linux users is well known: it has decent Microsoft Office support (although I hardly ever need it; most users don’t mind exporting the files in an open document format and publishers often support OpenOffice/LibreOffice formats themselves) and its features are becoming more and more powerful, such as the CMIS support (for online collaboration through content management systems). It also has a huge community, sharing templates and other documents that make life with LibreOffice even much prettier. Don’t forget to check out its extensive documentation.

The aspects of LibreOffice I use the most are of course its writer (word processor) and calc (spreadsheet application). The writer-part is for when I do technical writing, whereas the spreadsheet application is for generating simple management sheets for startups and households that want to keep track of things (such as budgets, creating invoices, data for mail-merge, etc.). At my work, Excel is one of the most used “end user computing” tools, so I happen to get acquainted with quite a few spreadsheet tips and tricks that are beneficial for small companies or organizations ;-) Also, Calc has support for macro-like enhancements, which makes it a good start for fast application development (until the requests of the user/client has been stabilized, after which I usually suggest a real application development ;-)

I generally don’t use its presentation part much though – if I get a powerpoint, I first see if Google Docs doesn’t show it sufficiently well. If not, then I try it out in LibreOffice. But usually, if someone sends me a presentation, I tend to ask for a PDF version.

Last week Google announced all the accepted student proposals for Google Summer Of Code 2013… included mine !!
I will participate in this year’s GSoC with Gentoo Foundation. My proposal was about identity.gentoo.org (aka Gentoo Identity). Gentoo Identity will be an LDAP web interface written using Django, which is a Python web framework. That will allow Gentoo developers _and_ users to configure easily their attributes from Gentoo’s LDAP server. This project is based on a previous GSoC project(codename ‘Okupy’). The web application will be for general usage and not only for LDAP administrators and sysadmins. I am also going to rewrite perl_ldap script  in python and improve it.  Currently perl_ldap  script is the only way to edit attirbutes from Gentoo’s LDAP server.

Some major features are:

• LDAP Authentication.
• Editable LDAP attributes via web forms.
• Information about Gentoo developers (full name, gpg key, location, team) in a list like a address book.
• Additional information about other accounts based on the ACL.
• Enable privileged users/groups(recruiters, devrel, infrastructure team) to add new accounts and edit their information based on their ACL.

For more information about Gentoo identity, check my full proposal.

The expected outcome is a fully functional and scalable LDAP web interface, where both users and developers will be able to edit their attributes easily.

Theo Chatzimichos from Gentoo Infrastructure team will be my mentor and my co-mentor will be Gentoo Infrastructure lead, Robin Johnson. Also Gentoo dev, Matthew Summers as  a previous GSoC mentor of Okupy will help us in the Django part. In addition Michał Górny will work in parallel in another GSoC project for Gentoo Identity. His project aims to built a complete OpenID provider on top of it, to provide a common authentication and identity exchange mechanism for all Gentoo sites.

I am going to post weekly reports about the status of the project with more technical details. To come in contact for feedback and questions please send your emails at identity[AT]gentoo[DOT]org.

Happy coding!

Recently I had some time to do some clenaups/changes/updates in server:database repo regarding MySQL (and MariaDB). Nothing too big. Well actually, there are few little things that I want to talk about and that is the reason for this blog post, but still, nothing really important…

MySQL 5.5, 5.6 and 5.7

MySQL 5.6 is stable for some time already, so it’s time to put it in the action. So I sent the request to include it in Factory and therefore in openSUSE 13.1. There is off course a list of interesting stuff you might want to take a look at before you update. If you don’t want to update, you can install mysql-community-server_55 from server:database repo and stay a little bit longer with version 5.5. On the other hand, staying with old versions is boring, so you can also switch to mysql-community-server_57 which provides new MySQL 5.7. So if you are into databases and especially into MySQL and forks (we have MariaDB 5.5 and 10.0 as well), we have plenty of toys for you to play with.

NOTE: Having MySQL 5.6 in openSUSE 13.1 doesn’t mean switching default back to Oracles MySQL, M in LAMP still means MariaDB for whatever it is worth. It just mean, that you have MySQL 5.6 as an alternative available if you prefer it.

Default configuration

One of the interesting changes that happened in MySQL 5.6 is new default configuration. MySQL usually shipped with some examples of configuration that you can use. It  was there since forever and never changed, although typical computers went from 256M of RAM to 8G. It contained some buffers sizes and various other optimizations. I heard various complains that it would be better shipping without it than with the one that is there. What folks at Oracle did was drop most of it and replace it with pretty much empty one, with various settings commented and described. They probably heard the same complains I consider it a really good step. Defaults are bult-in after all, so why to put them in config file? So I took theirs, added few things. For example Barracuda file format. It was set to be default upstream for few versions but they decided to go back to Antelope. But it’s also one of the thing people complain to me the most about – that they have to set file_per_table and Barracuda manually. And I added examples for multi configuration that we for some reason have included and exposed. This same config file will be pushed to MariaDB as well.

If you are interested in current state, you can see the config file on github and if you have some suggestions that everybody can benefit from, let me know either via comments or via pull request on github

 

Browsers are becoming application disclosure frameworks rather than the visualization tools they were in the past. More and more services, like the Draw.io one I discussed not that long ago, are using browsers are their client side while retaining the full capabilities of end clients (such as drag and drop, file management, editing capabilities and more).

The browser I use consistently is Firefox. I do think I will move to Chromium (or at least use it more actively) sooner or later, but firefox at this point in time covers all my needs. It isn’t just the browser itself though, but also the wide support in add-ons that I am relying upon. This did make me push out SELinux policies to restrict the actions that firefox can do, because it has become almost an entire operating system by itself (like ChromeOS versus Chrome/Chromium). With a few tunable settings (SELinux booleans) I can enable/disable access to system devices (such as webcams), often vulnerable plugins (flash, java), access to sensitive user information (I don’t allow firefox access to regular user files, only to the downloaded content) and more.

One of the add-ons that is keeping me with Firefox for now is NoScript. Being a security-conscious guy, being able to limit the exposure of my surfing habits to advertisement companies (and others) is very important to me. The NoScript add-on does this perfectly. The add-on is very extensible (although I don’t use that – just the temporary/permanent allow) and easy to work with: on a site where you notice some functionality isn’t working, right-click and seek the proper domain to allow methods from. Try-out a few of them temporarily until you find the “sweet spot” and then allow those for future reference.

Another extension I use often (not often enough) is the spelling checker capabilities. On multi-line fields, this gives me enough feedback about what I am typing and if it doesn’t use a mixture of American English and British English. But with a simple javascript bookmarklet, I can even enable spell check on a rendered page (simple javascript that sets the designMode variable and the contentEditable variable to true), which is perfect for the Gorg integration while developing Gentoo documentation.

The abilities of a browser are endless: I have extensions that offer ePub reading capabilities, full web development capabilities (to edit/verify CSS and HTML changes), HTTPS Everywhere (to enforce SSL when the site supports it), SQLite manager, Tamper Data (to track and manipulate HTTP headers) and more. With the GoogleTalk plugins, doing video chats and such is all done through the browser.

This entire eco-system of plugins and extensions make the browser a big but powerful interface, but also an important resource to properly manage: keep it up-to-date, backup your settings (including auto-stored passwords if you enable that), verify its integrity and ensure it runs in its confined SELinux domain.

Okay, this just had to be here. I’m an automation guy – partially because of my job in which I’m responsible for the long-term strategy behind batch, scheduling and workload automation, but also because I believe proper automation makes life just that much easier. And for personal work, why not automate the majority of stuff as well? For most of the automation I use, I use bash scripts (or POSIX sh scripts that I try out with the dash shell if I need to export the scripts to non-bash users).

The Bourne-Again SHell (or bash) is the default shell on Gentoo Linux systems, and is a powerful shell in features as well. There are numerous resources available on bash scripting, such as the Advanced Bash-Scripting Guide or the commandlinefu.com (not purely bash), and specific features of Bash have several posts and articles all over the web.

Shell scripts are easy to write, but their power comes from the various tools that a Linux system contains (including the often forgotten GNU-provided ones, of which bash is one of them). My system is filled with scripts, some small, some large, all with a specific function that I imagined I would need to use again later. I prefix almost all my scripts with sw (first letters of SwifT) or mgl (in case the scripts have the potential to be used by others) so I can easily find them (if they are within my ${PATH} of course, not all of them are): just type the first letters followed by two tabs and bash shows me the list of scripts I have:

$ sw\t\t swbackup swdocbook2html swsandboxfirefox swletter swpics swstartvm swstripcomment swvmconsole swgenpdf swcheckmistakes swdoctransaccuracy swhardened-secmerge swmailman2mbox swmassrename swmassstable swmovepics swbumpmod swsecontribmerge swseduplicate swfindbracket swmergeoverlay swshowtree swsetvid swfileprocmon swlocalize swgendigest swgenmkbase swgenfinoverview swmatchcve $ mgl\t\t mglshow mglverify mglgxml2docbook mglautogenif mgltellabout mgltellhowto mgltellwhynot mglgenmodoverview mglgenoval mglgensetup mglcertcli mglcleannode mglwaitforfile

With the proper basic template, I can keep the scripts sane and well documented. None of the scripts execute something without arguments, and “-h” and “–help” are always mapped to the help information. Those that (re)move files often have a “-p” (or “–pretend”) flag that instead of executing the logic, echo’s it to the screen.

A simple example is the swpics script. It mounts the SD card, moves the images to a first location (Pictures/local/raw), unmounts the SD card, renames the pictures based on the metadata information, finds duplicates based on two checksums (in case I forgot to wipe the SD card afterwards – I don’t wipe it from the script) and removes the duplicates, converts the raws into JPEGs and moves these to a minidlna-served location so I can review the images from DLNA-compliant devices when I want and then starts the Geeqie application. When the Geeqie application has finished, it searches for the removed raws and removes those from the minidlna-served location as well. It’s simple, nothing fancy, and saves me a few minutes of work every time.

The kiss tools are not really a toolset that is called kiss, but rather a set of commands that are simple in their use. Examples are exiv2 (to manage JPEG EXIF information, including renaming them based on the EXIF timestamp), inotifywait (passive waiting for file modification/writes), sipcalc (calculating IP addresses and subnetwork ranges), socat (network socket “cat” tool), screen (or tmux, to implement virtual sessions), git (okay, not that KISS, but perfect for what it does – versioning stuff) and more. Because these applications just do what they are supposed to, without too many bells and whistles, it makes it easy to “glue” them together to get an automated flow.

Automation saves you from performing repetitive steps manually, so is a real time-saver. And bash is a perfect scripting language for it.

mongodb-2.4.4

Just bumped it to portage and fixed an open bug along. This is yet another bugfix release which backports the switch to the Cyrus SASL2 library for sasl authentication (kerberos). Dependencies were adjusted so you no longer need libgsasl on your systems (remember to depclean).

highlights

• config upgrade fails if collection missing “key” field
• migrate to Cyrus SASL2 library for sasl authentication
• rollback files missing after rollback

pymongo-2.5.2

This one is important to note and I strongly encourage you to upgrade asap as it fixes an important security bug (CVE-2013-2132). I’ve almost dropped all other versions from tree anyway…

highlights 2.5.x

• support GSSAPI (kerberos) authentication
• support for SSL certificate validation with hostname matching
• support for delegated and role based authentication

mongodb-2.5.x dev

What’s cooking for the next 2.6 releases ? Let’s take a quick look as of today.

• background indexing on secondaries (hell yes!)
• new implementation of external sort
• add support for building from source with particular C++11 compilers (will fix a gentoo bug reported quite a long time ago)
• mongod automatically continues in progress index builds following restart

In the past, when I had to manage my images (pictures) I used GQview (which started back in 2008). But the application doesn’t get many updates, and if an application does not get many updates, it either means it is no longer maintained or that it does its job perfectly. Sadly, for GQview, it is the unmaintained reason (even though the application seems to work pretty well for most tasks). Enter Geeqie, a fork of GQview to keep evolution on the application up to speed.

The Geeqie image viewer is a simple viewer that allows to easily manipulate images (like rotation). I launch it the moment I insert my camera’s SD card into my laptop for image processing. It quickly shows the thumbnails of all images and I start processing them to see which ones are eligible for manipulations later on (or are just perfect – not that that occurs frequently) and which can be deleted immediately. You can also quickly set Exif information (to annotate the image further) and view some basic aspects of the picture (such as histogram information).

Two features however are what is keeping me with this image viewer: finding duplicates, and side-by-side comparison.

With the duplicate feature, geekie can compare images by name, size, date, dimensions, checksum, path and – most interestingly, similarity. If you start working on images, you often create intermediate snapshots or tryouts. Or, when you start taking pictures, you take several ones in a short time-frame. With the “find duplicate” feature, you can search through the images to find all images that had the same base (or are taking quickly after each other) and see them all simultaneously. That allows you to remove those you don’t need anymore and keep the good ones. I also use this feature often when people come with their external hard drive filled with images – none of them having any exif information anymore and not in any way structured – and ask to see if there are any duplicates on it. A simple checksum might reveal the obvious ones, but the similarity search of geeqie goes much, much further.

The side-by-side comparison creates a split view of the application, in which each pane has another image. This feature I use when I have two pictures that are taken closely after another (so very, very similar in nature) and I need to see which one is better. With the side-by-side comparison, I can look at artifacts in the image or the consequences of the different aperture, ISO and shutter speed.

And the moment I start working on images, Gimp and Darktable are just a single click away.