Wednesday, April 28, 2021

Installing native Docker on AlmaLinux

 

Today, I had to reinstall a Docker host, which was previously running on my beloved CentOS 8. As that is quickly approaching it's end, I decided to give AlmaLinux a try. To use it as a Docker host, I had to install native Docker. (I prefer to use native Docker over the one, that is part of the distribution.) So, according to https://get.docker.com/, this is supposed to work as follows:


    curl -fsSL https://get.docker.com -o get-docker.sh
    sudo sh get-docker.sh

In the past, I did that on CentOS, quite a lot, so I expected it to work out of the box. Unfortunately, that didn't quite work, and failed with the error message below:


    sudo sh get-docker.sh
# Executing docker install script, commit: 7cae5f8b0decc17d6571f9f52eb840fbc13b2737 ERROR: Unsupported distribution 'almalinux'


In other words, the Docker installer isn't quite uptodate. Fortunately, fixing a shell script isn't that big of a problem, and I got it running by applying the following patch:

[jwi@gitjndhost ~]$ diff -ub get-docker-orig.sh get-docker.sh --- get-docker-orig.sh 2021-04-28 12:10:12.477498011 +0200 +++ get-docker.sh 2021-04-28 12:03:23.300011495 +0200 @@ -342,7 +342,7 @@ esac ;; - centos|rhel) + centos|rhel|almalinux) if [ -z "$dist_version" ] && [ -r /etc/os-release ]; then dist_version="$(. /etc/os-release && echo "$VERSION_ID")" fi @@ -427,8 +427,8 @@ echo_docker_as_nonroot exit 0 ;; - centos|fedora|rhel) - yum_repo="$DOWNLOAD_URL/linux/$lsb_dist/$REPO_FILE" + centos|fedora|rhel|almalinux) + yum_repo="$DOWNLOAD_URL/linux/centos/$REPO_FILE" if ! curl -Ifs "$yum_repo" > /dev/null; then echo "Error: Unable to curl repository file $yum_repo, is it valid?" exit 1

Note: In order to overcome minor dependency conflicts with podman, buildah, and the like, I also had to issue the command


    sudo dnf -y remove runc






        

Saturday, October 24, 2020

Incompatibility of Process.waitFor() between Linux, and Windows.

 

Today, after maybe two years, I figured out, why one particular piece of code works excellent on Windows, but has been causing a lot of trouble on Linux. That piece of code is the class Executor in my Java application framework library afw.


The reason turned out to be, that the method Process.waitFor() behaves differently on Windows, and Linux, with regard to the launched processes output.


  • On Windows, the waitFor() method waits, until
    1. the launched process has terminated, and
    2. the processes standard output, and error output has been consumed (in other words: The input streams, as returned by Process.getInputStream(), and Process.getErrorStream() have been read.
This behaviour can have the unexpected result, that the waitFor() method never returns, which can easily be dealt with by simply launching two separate threads, that read those input streams.
 
  • On Linux, however
    1. The method waits, again, until the launched process has terminated, but
    2. It doesn't wait for the consumption of the processes output.
 
In other words: If you are interested in the processes output, then it is not sufficient, to invoke Process.waitFor(), because the expected output may arrive later on. Instead, you need to launch the same two threads, and then wait, until
  1. both threads have received EOF from their respective input streams, and
  2. the invocation of Process.waitFor() has returned.
That's a bit tricky, indeed.
 
In summary: Java is a highly portable platform. That being said, there might still be issues. Glad, that I could clarify this one.
 

Sunday, February 3, 2019

Announce: JSGen (Java Source Generation Framework)


It is with some pride, that I can announce JSGen today, as a new, and very minor, open source project, under the Apache Software License, 2.0

JSGen is short for Java Source Generation framework. As the name (hopefully) suggests, it is a tool for generating Java source code. It is the result of close to 20 years working on, and with Java source generation. That includes, in particular, the XML data binding framework JaxMe, its predecessor Apache JaxMe, and a lot of application programming in my professional work.

It is my opinion, that


  1. Java source generators have a tendency to grow into CLOB's over time, becoming less maintainable, and understandable.
  2. Java source generators do typically contain a real lot of boilerplate code, organizing things like import lists, and syntactical details, rather than their actual purpose.


In contrast, JSGen provides an object model, that aims to make source generation just yet another developers task, on which modern software engineering principles can be applied. JSGen will support you by


  • Creating import lists semiautomatically (with the exception of static imports)
  • Supporting multiple formatting code styles (An Eclipse-like format, the default, and an Apache-Maven-like alternative format.) Switching between code styles is as easy as replacing one configuration object with another.
  • Enabling a more structured approach to source code generation. (Example: Implement the case of handling a single instance, and use that to handle the case of a collection.)


Let's have a look at an example, which is quoted from the JUnit tests. We intend to create a simple HelloWorld.java. Here's, how we would do that with JSGen.

    JSGFactory factory = JSGFactory.create();
    Source jsb =
        factory.newSource("com.foo.myapp.Main").makePublic();
    Method mainMethod =
        jsb.newMethod("main").makePublic().makeStatic();
    mainMethod.parameter(JQName.STRING_ARRAY, "pArgs");
    mainMethod.body().line(System.class,
                           ".out.println(",
                           q("Hello, world!"), ");");
    File targetJavaDir = new File("target/generated-sources/mysrc");
    FileJavaSourceWriter fjsw = new
       FileJavaSourceWriter(targetJavaDir);
    // You might prefer MAVEN_FORMATTER in the next line.
    fjsw.setFormatter(DEFAULT_FORMATTER);
    fjsw.write(factory);

Some things I'd finally like to note:


  1. JSGen is a complete rewrite of two mature, and reliable predecessors, namely JaxMeJS, and Apache JaxMe JS. As such, it is based on a mature API, and a real lot of applied experience. It may be new, but it should be quite reliable, nevertheless.
  2. It provides all the features, of the predecessors, but adds
    1. Support for Generics
    2. Support for Annotations
    3. Switchable Code formatters
    4. Builders heavily used in the API

This announcement has intentionally been deferred until the point, where JSGen has succesfully been used for a professional project in my professional work. 

Finally, a few links:



  

Wednesday, February 24, 2016

Don't use Copy+Paste on Windows, if you've got a lot of data

Windows never ceases to disappoint me. I've got an external hard drive, on which resides a 60 GB VM, which I need to copy now and then to emulate a snapshot. So far, I used the standard Windows file copy stuff to do that. (Copy, and paste the VM directory.) Ran with about 8-20 MB per second. Or in other words: Took up to two hours. And, worst of all, the procedure isn't reentrant. Interrupting the copying means to restart.

So, tried something different: CygWin's rsync, the swiss army knife for backups, and related stuff. In other words, I am using the command

  rsync -a -r --progress
Runs with 30MB per second (50% faster than native Windows copying). And is reentrant...

Thursday, November 5, 2015

VMWare Shared Folders oin Fedora 23


Hi,

if you haven't noticed: Fedora 23 is out, and its time to get your hands wet on it. Being forced to use a Windows Laptop, I am typically doing this by installing a VMware guest. Once the virtual machine is up, the first thing i want to do is accessing my Windows home directory. The easiest way to do this is by creating a so-called "Shared Folder" in the VMware settings for my machine. On the guest, you need to install the "VMware Tools" and do something like

mkdir /home/username/sharedfolder
sudo mount -t vmhgfs -o uid=1000,gid=1000 .host:sharename /home/username/sharedfolder

(The options uid=1000,gid=1000 ensure, that the mounted directory is readable, and writable for the user with uid=1000, and gid=1000, which is me.)

The problem with that procedure is, that it depends on a kernel module called "hgfs", which must be installed as part of the "VMware Tools". And, needless to say: Installation of the VMware Tools (version 9.2.0-799703, as of this writing) fails, because the Kernel isn't compatible to the sources distributed by VMware.

So far, the only reasonable solution was to wait for an updated tools version by VMware. (I generally ignored the possibility to patch those distributed sources as overly complicated, and insecure. However, there's a new, and better solution available:

Fedora 23 automatically installs an RPM named open-vm-tools. And this includes two programs, that allow use of shared folders without the kernel module:

  # Display a list of all shared folders:
  # Note, that it includes a share called "sharedfolder".
  $ vmware-hgfsclient sharedfolder

  # Create a directory named /home/username/foo, and mount the shared folder there.
  $ vmhgfs-fuse .host:foo /home/username/sharedfolder
  $ cd /home/username/sharedfolder
  $ touch testfile
  $ rm testfile

Note, that neither "sudo" nor the specification of any options was required.

So, in other words: As of Fedora 23, the VMware Tools are no longer required. Mouse integration, Cut and Paste, and using shared folders: Everything works out of the box. (I can live without the thin print drivers.)




Tuesday, March 17, 2015

Honouring Terry Pratchett: Java Web Application

This Blogs name is Grumpy Apache. These days, there are excellent reasons, for being grumpy: After all, my favourite author Terry Pratchett has died, way too early. But, as Terry wrote in Going Postal about John Dearheart:

His name, however, continues to be sent in the so-called Overhead of the clacks. The full message is "GNU John Dearheart", where the G means, that the message should be passed on, the N means "Not Logged" and the U that it should be turned around at the end of the line. So as the name "John Dearheart" keeps going up and down the line, this tradition applies a kind of immortality as "a man is not dead while his name is still spoken".
This means, we'll be celebrating "Being childish day" today, by adding the HTTP Header

      X-Clacks-Overhead: GNU Terry Pratchett

to our web sites. And, here's how to do that with any Java Web Application.

Its simple. First of all, you'll be adding this class to your web application. It is a so-called servlet filter. I'll quote the relevant method here:

public void doFilter(ServletRequest pReq, ServletResponse pRes,
                                         FilterChain pChain) throws IOException,
ServletException {
if (pRes instanceof HttpServletResponse) {
final HttpServletResponse res = (HttpServletResponse) pRes;
res.addHeader("X-Clacks-Overhead", "GNU-Terry-Pratchett");
}
pChain.doFilter(pReq, pRes);
}
Besides, add the following snippets to your web.xml:

 
    ClacksOverheadFilter
    com.github.jochenw.clacksov.ClacksOverheadFilter
 

 
    ClacksOverheadFilter
    *
 
And, that's it! No modification of servlets, or the like, just a simple addition, that you can make to any web application.

Keep in mind:

As long as we are shifting his name on the Internet, Terry isn't dead.



Tuesday, June 17, 2014

Installing CentOS 7 Prerelease on VMWare


Hi,

if you haven't heard the news: A prerelease of CentOS 7 is out. This is important, because:
  1. CentOS 7 is a major release and will be the base of the Linux Distro that people like me will be using in the next years on servers. (Yes, I do know about Ubuntu 14.04 LTS, OpenSUSE Whatever, Debian Something, etc. However, that is most likely not what I will be using. Logically, so won't do people like me. End of discussion.)
  2. Quite a few things have changed since CentOS 6. In particular, much has been adopted from recent Fedora versions: 
    1.  The new Anaconda Installer. (I am personally not overly happy with it. The old one worked quite well for me, but I had my share of trouble with the new one. In particular, I am less than enthusiastic about how Disk Partitioning works nowadays. OTOH, this version of Anaconda (the one distributed with the CentOS 7 Prerelease) is a step forward in that aspect. Perhaps, more people like me had similar trouble.)
    2.  GNOME 3: Well, this one will definitely be the cause of a major uproar on the Red Hat Continent. I readily admit that I was one of the people who initially went with MATE as a GNOME 2 replacement, so as to avoid GNOME 3. However, in the meantime, I've learned to live with it and can even appreciate some features like the enhanced keyboard control. The one thing I am still missing is the pictures screenblanker, though. I learned to live with xscreensaver, although this still smells like a very ugly hack.) Like it, or not, people like me (c) will have to face it.
  3.  This prerelease was published not even one weak after the release of RHEL 7. Compare that to the months we had with some minor versions of CentOS 6. So, we benefit from Red Hat adopting CentOS. Good news!
So, what is this posting about?

  •  I won't cover generic aspects of installing CentOS, or Fedora. I'll assume that you have installed either of which before and have a rough idea of what I am talking about. In particular, I assume that you know what a network installation is, because right now this is the only installation method available through an ISO image. (Forget "Live DVD", or whatever else you have hoped for.)
  • I will, hovever, concentrate on installing this very special prerelease version, because it is not quite like installing an official version. (Neither is it overly complex, though.) Hopefully, I'll also cover what has changed since version 6.
 If that is interesting for you: Read on. If not: I am sorry! Google (Planet Apache, or whatever else brought you here, did wrong and is to blame).

So, what's to do?
Download the ISO Image from http://buildlogs.centos.org/centos/7/os/x86_64-20140614/images/boot.iso and save it, for example as "centos7-netinstall.iso".
  1. Create a new VM (My Parameters were "I will install the operating system later.", "Guest operating system=Linux", Version="CentOS 64-bit", Maximum disk size=30GB, Memory=3072GB. Everything else was as suggested by VMWare Player 6.0.2 build-1744117.
  2. Select Virtual Machine Settings, CD/DVD (IDE). Enable "Connect at power on" and "Use ISO Image file". Select the file you downloaded in step 1.
  3. Start up the created VM. From the boot menu, select "Install CentOS 7". (You may as well test the media, but you did check the MD5 Sum anyways, did you? :-) At least, you know the difference... (Remember that "won't cover generic aspects" above?)
  4. Hopefully, the Anaconda graphical installer will come up. (At least, it does so on a VMWare machine. I'd never hope so on a machine with an NVIDIA or AMD graphics card. Don't expect me to help you with that crap.  I'm all with Linus on that. :-)
  5. Select your language (Safe choice is, of course,"English-US").
  6. Anaconda will notify that  this is prerelease, unstable software. You knew that anyways, so click on "I want to proceed."
  7. The Anaconda "Installation Summary" screen will come up. This will be an unknown thing (Remember: New Anaconda) for a lot of people, so here's a screenshot:


    The important thing to keep in mind is the order of the following steps.
  8.  Start with the Keyboard. (You're likely to use that in the following steps.) Click on "Keyboard" (Not the small keyboard icon, but the big icon, or the word.) Click on the "+" sign, and select your favourite keyboard layout. (In my case "German, Germany, Eliminate dead keys".) Remove any unwanted layout by clicking on it, and clicking on the "-" sign. Finish by clicking on "Done" in the upper left corner. (Who the heck came up with that? Anyways, remember the location.)
  9. The next thing you're gonna need is the network. (Most likely, you are currently "Not Connected".) Click on "Network & Hostname". Click on "Off" in the upper right corner to enable networking. Enter a meaningful host name. (I choose "c7wm96.mcjwi01.eur.ad.sag". Avoid "localhost.localdomain".) Click on "Done". (Upper left corner, remember?)
  10. Now we can edit "Date & Time", aka time zone. I choose "Europe/Berlin".
  11. If you need that (You don't, really...), click on "Language Support" and select additional languages.
  12. The most obvious trap is the "Installation Source" (Hopefully, it won't be in the official releases, which will select an URL automatically): Click on that, enable "On the network", and enter the URL buildlogs.centos.org/centos/7/os/x86_64-20140614/. If you need to use an HTTP Proxy, click on "Proxy setup". Enter your proxy host name and port (in my case "httpprox.hq.sag:8080") Click on "Add". Click on "Done". Wait a few seconds until you see "Downloading package metadata", or the like. If you do see something like "Error setting up Base Repository", changes are that the URL is wrong. Fix it, and retry. Wait a few seconds more until downloading the package data and checking for dependencies has finished.
  13. Next, go to "Software Selection". The default is "Minimal Install". This is fine, if you are happy with a server that has no X11 enabled. I choose "Server with GUI" instead, to make my colleagues happy. On the right hand side, you can choose to have KDE installed addizionally. (AFAIK, no support for MATE, Cinnamon, LXDE, whatever. No idea, whether that will come.) You might wish to deselect LibreOffice, if you manage to do that. Click on "Done". Wait a few seconds until the message "Checking for software dependencies" disappears.
  14. Another, somewhat difficult step is the "Installation Destination". Click on that. If you need "Custom Partitioning", enable "I will configure partitioning." below. (The default is "Automatically configure partitioning.", The presence of this option is what has changed since Fedora 20, and I consider this to be a major improvement.) Click on "Done", even if you're actually not. If the window for "Manual Partitioning" appears, select your desired partition type ("Standard Partition", "BTRFS", "LVM") and add a few partitions by clicking on the "+" button. I create the following partitions (in that order):
    1. /boot with a Capacity of 500MB.
    2. Swap with a Capacity of 6GB. (I need that much, because the Oracle Installer wants 8GB of physical memory, but accepts Swap as a replecement.)
    3. / with a Capacity of 8 GB.
    4. /home with a Capacity of 16.21GB
    Click on "Done". Click on "Accept Changes". If no error messages can be seen on the "Installation Summary" screen, then you have mastered the major hurdles.
  • Click on "Begin Installation".
  • Regardless of the ongoing installation, click on "Root Password". Enter a meaningful, and secure, root password. Repeat it. Click on "Done". (You never even considered to enter a weak password, did you? Well, if you did: Click on "Done" twice. :-)
  • The installation is still ongoing. Click on "User Creation". Enter a real name and a login name, enable "Make this user administrator" (The option will actually add the created user to the "wheel" group, which has permissions to use "sudo"). Enter a password and repeat it. Click on "Done" twice. (Oops, your passsword is secure: Then once is sufficient.)
  • Keep in mind that this is a "network installation": Anaconda will download each and every single RPM to install (In my case about 1200.), so the process will take time. OTOH, with a fast network (DSL, or something like that) it won't take much longer than installation from a DVD.
  • Once the actual installation is finished, you'll be asked for a reboot. Confirm that, and the new system comes up. Almost done. One minor step to perform: Accept the GPL license, and accept another reboot. (No, this isn't Windows, but still....)
If you got this far, then you've got a system running CentOS 7 Prerelease. Congratulations. Unfortunately, one thing is still left. Your system doesn't have a valid Yum configuration. (Convince yourself by running
  sudo yum repolist all
Oops, you need a terminal window to do that. That's no problem if you are running KDE or any other desktop that you are used to. If it's GNOME 3, and you are not, here's what to do: Press, and release, the "Windows" key. (No, this is still not Windows, but anyways. If it helps, call it the "Linux" key.) Press, and release, the following keys, in that order: "t", "e", "r", "m", and Enter. At that point, a GNOME Terminal window should appear. (Or, in theory, any other desktop application containing the word "term". However, you had no chance to install "xterm" do far. :-) Using the command
  sudo yum vi /etc/yum.repos.d/centos7-prerelease.repo
create a new file with the following contents:
  [centos7-prerelease]
  name=CentOS 7 Prerelease
  url=http://buildlogs.centos.org/centos/7/os/x86_64-20140614/
  enabled=1
  priority=1
  gpgcheck=0
And now (I am not avoiding any flame wars today :-) you can do
  sudo yum install emacs emacs-nox gcc make binutils kernel-headers
A final note on the VMware tools: Anaconda did automatically install "open-vm-tools-desktop". So, mouse integration, copy and paste, etc. worked immediately for me. No need for a seprate installation.