Christians Tagebuch Introducing phorkie: git based pastebin (18.4.2012, 20:22)

At work, we've been looking for a pastebin application with a special feature set:

  • Self-hosted, so we can paste code and data without having to care about sensitive customer data becoming public accidentially
  • written in PHP, since we're a PHP shop
  • editability so other people can do fixes for us
  • title support for pastes so we can find it later
  • navigation among pastes so we don't have to remember the number later
  • search because navigation doesn't help with 1k+ pastes

We did not find anything suitable after researching for some days. Since I needed a new spare time project, I began working on phorkie.

After working on it for a month, I've just released the first version 0.1.0 and you're welcome to try it.

phorkie

phorkie is a self-hosted pastebin software written in PHP. Pastes are editable, may have multiple files and are stored in git repositories.

It turned out to be a clone of github's gist tool, but with less features - for now.

Features

  • Every paste is a git repository
    • Repositories can be cloned
    • Clone url can be displayed
  • Multiple files in one paste
  • Pastes can be edited
    • Add and delete files
    • Replace file with upload
    • History in sidebar
    • Easy access to old revisions
  • Syntax highlighting with GeSHi
  • rST rendering with rst2html (python docutils).
  • image upload and display
  • external tool support (xmllint, php syntax check)

Missing tools

For the history/log display, I wanted to display the timestamps in a human readable manner, e.g. "5 minutes ago" instead of 2012-04-18 22:36.

Looking for existing libraries, I found plenty for Python and Javascript, but not a single one for PHP. After asking on stackoverflow without getting a usable answer, I wrote my own and proposed it to PEAR: Date_HumanDiff.

Screenshots

new paste paste list history tools

Also see the wiki.

Code

The code is in a git repository on sourceforge but also mirrored on github for your convenience.

Link
Codelog Quickest way to split up multiple pear packages into different git repositories (11.4.2012, 20:09)

I was doing it the hard way before, but now:

$ git clone git@github.com:pear/Validate.git

# Go hit https://github.com/organizations/pear/repositories/new to make a new repo

$ git clone git@github.com:pear/Validate_CA.git

$ cd Validate_CA

$ git pull git@github.com:pear/Validate.git

$ git rm -r *

$ pear list ../Validate/package_CA.xml | grep ".* /" | cut -f 1 -d " " | xargs git reset HEAD

$ pear list ../Validate/package_CA.xml | grep ".* /" | cut -f 1 -d " " | xargs git checkout --

$ git commit -m "Splitting off to own package"

$ git push -u origin master

$ cd ..

$ cd Validate

$ pear list package_CA.xml | grep ".* /" | cut -f 1 -d " " | xargs git rm

$ git rm package_CA.xml

$ git reset HEAD LICENSE

$ git checkout -- LICENSE

$ git commit -m "Splitting off to own package"

$ git push

Link
Chuck Burgess DocBlox Is Unmasked ... It Is Really phpDocumentor 2 ! (16.3.2012, 13:10)

Link
till's blog Deploying PHP applications: PEAR and composer resources for chef (24.2.2012, 16:51)

This is something experimental I have been working on for our chef deployments. So the objective was/is to find a sane way to install PEAR packages and install dependencies with composer.

execute in chef recipes

In chef recipes, almost everything is a resource. In case you're just getting started with Chef, a list of current resources is available on the Opscode Wiki. It's a link I put in my browser bar since I frequently work on chef recipes.

Some examples for resources are:

  • package (to install software)
  • cron (setup a crontab)
  • directory (create directories)
  • template (install customized configuration files)
  • user and group (to create users and groups)
  • mdadm (to setup a RAID)

The above list are examples — so there is more. But if there isn't a designated resource, you can always use an execute block.

An example for an execute block is the following:

execute "discover a pear channel" do
  command "pear channel-discover easybib.github.com/pear"
end

This works pretty well, but it is also not very robust.

Fail hard

By default whenever a command fails, chef fails hard.

To illustrate what I'm talking about, let's test and execute the command from our execute block multiple times on the shell to see its exit status ($?):

till:~/ $ pear channel-discover easybib.github.com/pear
Adding Channel "easybib.github.com/pear" succeeded
Discovery of channel "easybib.github.com/pear" succeeded
till:~/ $ echo $?
0
till:~/ $ pear channel-discover easybib.github.com/pear
Channel "easybib.github.com/pear" is already initialized
till:~/ $ echo $?
1

So whenever a command returns not 0, chef will bail.

One solution is to brute-force your way through these things with ignore_failure true in your execute block. But that's usually not a great idea either because it hides other issues from you (and me) when we need to debug this later on.

For example, if this PEAR channel is unavailable during your next chef-run, it would be much, much harder to find the root cause as of why the install commands failed.

Another solution is using the not_if or only_if options with execute:

execute "discover a pear channel" do
  command "pear channel-discover easybib.github.com/pear"
  not_if do
    `pear channel-info easybib.github.com/pear`
  end
end

If the command wrapped in not_if succeeds (success is exit status), we would skip the execute block.

Optimize?

Since I discovered not_if and only_if, it allows me write recipes which work in most cases. More robust code, which allows me to re-execute recipes on already running instances. So for example when I update a recipe or configuration file which is distributed through a recipe I can re-run the entire recipe and it will not fail but instead complete successfully.

One problem remains with this approach I end up doing the same checks again and again.

... more after the jump.

Link
Christians Tagebuch WebFinger library for PHP released (24.2.2012, 09:12)

I'm implementing OpenID for SemanticScuttle, your self-hosted social bookmark manager. To log in with OpenID, you need to know your OpenID URL, which many people do not know, and don't want to know. Most know their email address, and thanks to WebFinger, this is all you have to know!

WebFinger enables applications to discover information about people by just their e-mail address - for example their OpenID URL!

I didn't find a single standalone WebFinger library for PHP, so I asked on StackOverflow, but did not get any responses. Failed to stand on the shoulders of giants, I went the hard way and implemented it all myself: Net_WebFinger, based on XML_XRD.

Implementation

WebFinger weaves RFC 6415: Web Host Metadata with LRDD which both use XRD files.

Thus the first step was to build a clean XRD library for PHP, with an intuitive API and 100% unit test coverage. I proposed the XML_XRD package on 2012-02-01, called for votes 8 days later. It was accepted with 11 votes. Extensive documentation does also exist now.

After the foundation was laid, I proposed the Net_WebFinger package. It was accepted as new PEAR this night, and just some minutes ago it got its first official release and a lot of documenation.

Usage

So, discovery is easy now! First, install the PEAR package:

$ pear install net_webfinger-alpha

Now the PHP code:

finger('user@example.org');
if ($react->openid !== null) {
    echo 'OpenID provider found: ' . $react->openid . "\n";
}

//list all other links:
foreach ($react as $link) {
    echo 'Link: ' . $link->rel . ' to ' . $link->href . "\n";
}
?>

WebFinger CLI

Net_WebFinger ships with a command line client that you can use to try it out. Find it with

$ pear list-files net_webfinger|grep cli
doc  /usr/share/php/docs/Net_WebFinger/examples/webfinger-cli.php

Yahoo and Google already support WebFinger. Distributed social networks like status.net (that powers identi.ca) and Diaspora use WebFinger to distribute public encryption keys, OStatus and Salmon URLs. You can try one of those user addresses, too.

$ php /usr/share/php/docs/Net_WebFinger/examples/webfinger-cli.php klimpong@gmail.com
Discovering klimpong@gmail.com
Information secure? false
OpenID provider: http://www.google.com/profiles/klimpong
Link: http://portablecontacts.net/spec/1.0: http://www-opensocial.googleusercontent.com/api/people/
Link: http://portablecontacts.net/spec/1.0#me: http://www-opensocial.googleusercontent.com/api/people/102024993121974049099/
Link: http://webfinger.net/rel/profile-page: http://www.google.com/profiles/klimpong
Link: http://microformats.org/profile/hcard: http://www.google.com/profiles/klimpong
Link: http://gmpg.org/xfn/11: http://www.google.com/profiles/klimpong
Link: http://specs.openid.net/auth/2.0/provider: http://www.google.com/profiles/klimpong
Link: describedby: http://www.google.com/profiles/klimpong
Link: describedby: http://www.google.com/s2/webfinger/?q=acct%3Aklimpong%40gmail.com&fmt=foaf
Link: http://schemas.google.com/g/2010#updates-from: https://www.googleapis.com/buzz/v1/activities/102024993121974049099/@public
$ php /usr/share/php/docs/Net_WebFinger/examples/webfinger-cli.php singpolyma@identi.ca
Discovering singpolyma@

Truncated by Planet-PEAR, read more at the original (another 1043 bytes)

Link
PEAR What would you do with 5 million lines of code? (24.1.2012, 14:05)

Since October 2011, 5 million lines of the PEAR codebase has shifted to github.

Hand in hand with this shift has been the tireless work of Daniel C – someone who brazenly said “I will fix the failing packages!” in the tail end of last year.

Coupling his efforts with a call to arms, we’ve now seen an evaluation of the Known Good packages against PHP 5.4, and massive input by the community. The net result is as follows:

  • Releases of Text_LanguageDetect, HTTP2, Net_Growl, Image_QRCode, Tree, HTML_BBCodeParser, Net_IMAP, Net_DNSBL, Services_Amazon, Image_Barcode2, Validate, Console_Color2, Services_ExchangeRates, Validate_DK, PEAR_PackageFileManager_Frontend, Text_Highlighter, PHP_Shell, Date, Image_Text, PEAR_Frontend_Gtk2, PHP_DocBlockGenerator, & Validate_AR through Dec/January
  • All test infrastructure upgrading to PHP 5.4 release candidates
  • All database driven test suites executing properly, catching a variety of simple bugs
  • Just shy of 900 commit emails to the pear-cvs list for Dec/Jan – many containing multiple commits & fixes
  • Hitting a point of “near zero” patches to be applied to unmaintained packages
  • Applying no less than 30+ patches contributed by the community across all of PEAR
  • Increasingly, the PEAR QA team is delivering PHP 5.3+ friendly forks of existing packages

I’d like to thank Daniel C for his efforts to date, as well as the contributors who may have previously lurked or found themselves distracted by other concerns.
Dec/Jan has been a great and vigorous period for the project – I heartily look forward to a great 2012.

Link
Christians Tagebuch Suhosin: Include filename .phar is not allowed (21.12.2011, 18:02)

When trying to run the PEAR installer go-pear.phar on one of our servers, it exited without any notice and without seeming to do anything.

After some searching I found that the syslog contained the error message:

Dec 15 12:59:47 scms suhosin[13658]:
  ALERT - Include filename ('phar://go-pear.phar/index.php')
  is an URL that is not allowed
  (attacker 'REMOTE_ADDR not set', file '/root/go-pear.phar', line 1236)

Dec 15 12:59:47 scms suhosin[13658]:
  ALERT - script tried to disable memory_limit by setting it to a negative value
   -1 bytes which is not allowed
  (attacker 'REMOTE_ADDR not set', file 'unknown')

So I knew it was Suhosin, the "advanced protection module for PHP5". It's enabled by default on Debian and Ubuntu.

Instead of simply disabling it, I allowed the phar stream wrapper with Suhosin:

$ emacs /etc/php5/cli/conf.d/suhosin.ini
.. add the following line:
suhosin.executor.include.whitelist = phar

Link
PEAR Welcome to new contributors (18.12.2011, 11:54)

With the PEAR move to github surpassing 200 repositories, we’re seeing more contributions from folks lurking in the shadows.

In particular I’d like to highlight the efforts of meldra and Gemorroj.
With XML_Feed_Parser hosted on github, Meldra has been able to provide all of the patches that have been sitting in the wings internally back to PEAR, with no fuss.
Faced with a backwards compability requirement on Image_Barcode, Gemorroj contributed heavily to an Image_Barcode2.

Having watched these two individuals over the last few weeks provide new vigour and input to some of our underloved packages, I’d like to put a challenge out to the community.

If you have a patch we have pushed back on because of backwards compatibility concerns, talk to us about making the next significant version of that package – we’ll get the code on github and help you get what you need.

No red tape. No run around. Just a solution to your problem by creating an appropriate fork, and a new major version to avoid any BC concerns.

If you have fixes for defects or enhancements being used within your organisation – send us a pull request.

Where there isn’t source available on github yet – ask for it.

PEAR is about providing the PHP community with reusable, effective components – this has been our mission since day 1.
If there is anything we can do to make that goal happen, to assist you as an individual or company, I would strongly encourage you to let us know – we’re here to help.

Link
Codelog Managing multiple job configurations for Jenkins (13.11.2011, 04:40)

If you are in the same boat as I am, you find you have too many packages to look after with Jenkins.

The beauty of Jenkins is the simplicity at setting up a job with the web frontend - but once you get over a certain level of complexity this is actually one of the bigger drawbacks.

Sure, we've got some templates, but how far can you really stretch it?

In my situation, I need to:

  1. Trawl SVN/other version control for all packages available - several hundred
  2. Only if the package has tests, add an entry to the CI suite
  3. Adapt to packages which require E_ALL & ~E_STRICT to run happily under that
  4. Packages which require dependencies, but can't be installed, still need a mechanism to install said dependencies
  5. And some which need to be invoked with the legacy AllTests.php
  6. Detect when a package has migrated to github
  7. ... and update an existing build/job with a new tool when required
I had tackled part 1 with pear's "packages-all" SVN link, which pointed to the trunk branches of all relevant code, and written some scripts for cruisecontrol to find all directories with a /tests/, but I find myself in need of something more.

So, my code is on github for now, and you can see the current CI system where those scripts have installed new jobs.

I'm quite sure that pyrus and a local installation will deal with the dependencies; as they are all described with PEAR's package.xml format. Also; detecting when a package has shifted to github should be fairly easy to tackle, as there is much work underway to deal with migration.

The one area I need to explore is manipulating jenkins jobs via xpath, to understand what parts of a job are already present and what need updating - basically number seven in the above list.

I'm curious who's done this sort of thing before, regardless of language, and if there are any libraries which make it easier to do this sort of thing.


Link
PHP_CodeSniffer development moved to Git (7.11.2011, 23:45)

Quite a lot of PEAR packages are being moved to hosting on Github and Squiz Labs is starting to host projects over there as well so I've moved development of PHP_CodeSniffer to Github and will be closing down the existing SVN repository. The major benefit of this change is...

Link
Links RSS 0.92   RDF 1.
Atom Feed  
PHP5 powered   PEAR
Link the Planet <a href="http://www.planet-pear.org/">Planet PEAR</a>