Backing up Solr from a Linux to Windows

Posted on December 5, 2012 by Eva

I’ve come across a problem with backing up Solr. Before 3.6 if you try to back it up you will have to unload the core or stop the server that hosts Solr before backing up the files.

Even with 3.6 and later, while the backup feature does exist which allows you to backup using the replication handler to get a snapshot created in a directory of your choice (location parameter) the snap shot is not differential so it will just copy your entire index and there is no way to tell if the backup has completed successfully or is in the middle of backing up. It creates a new lock file in the source code but never obtains it, so a lock file is never created.

Plus you would have to write your own retention script as the one supplied only handles “number of backups to retain” which isn’t great if you have a 200GB Solr index and you want to hold onto the backups for a week and if you have one central backup server then it would have to ship the backups to that server, which can be a little awkward if, as is in my case, your backup server is running windows.

Ideally you would have an application that calls the replication handler and asks it for the current index just like a Solr slave. You could do differential backups, resume when the connection falls over, put it in a directory of your choice and even back up all the configuration files all in the same simple system.

I’ve written a c# program for this in work, which means I can’t release the source, but perhaps it might be better to write in Java anyway.

Backup is important and I think it’s worth addressing, especially since a lot of Solr indexes can take a lot of time to recreate.

Rating: 10.0/10 (1 vote cast)

The Pomodoro Technique Part 1

Posted on December 5, 2012 by Eva

For those of you who have me in your RSS, I’m sure your surprised to see my blog reappear. I’ve taken on a new challenge to start talking about any work I do, I hope it will enhance my career and encourage me to do more of it. The problem as per usual is that I don’t have the motivation to do much after work and the more I consider how to resolve such an issue the more I find myself becoming anxious. So in a new effort to combat that I’ve started trying the Pomodoro Technique.

The basic concept is this:

List the tasks that you have to complete
Add estimates for these tasks in terms of Pomodoros (25 minute periods)
Start the tasks and take a 5 minute break every 25 minutes.

I assume that you should take longer breaks the more pomodoros you do.

I have found it works well at home but I have yet to give it a full chance at work yet due to these reasons:

What do I do during 5 minute breaks? What is my boss going to think if I leave my desk?
I get interrupted all the time in work, colleague needs help with something or a manager needs a question about the system answered
A lot of the tasks I do take less than 25 minutes, how do they fit in?

Here is the website for the pomodoro technique http://www.pomodorotechnique.com/
A book which I have yet to read is available on their site.

I was using a pomodoro timer on my computer, you can get that here: http://code.google.com/p/pomodairo/

I’ll add new parts to this post as I explore this further.

Rating: 10.0/10 (1 vote cast)

phpBB youtube BBCode

Posted on December 4, 2012 by Eva

The standard response to how do I create a YouTube BBCode is

Use the following BBCode usage:

[youtube]http://www.youtube.com/watch?v={SIMPLETEXT}[/youtube]

Then use the following HTML Replacement:

Of course when you try this you will run into a problem. When your users attempt to try it with a YouTube url they copy pasted from YouTube they may be using https or there may be extra query parameters which will mean that the YouTube simply won’t work.

A quick change will allow you to support that issue and as an added bonus all YouTube references will automatically use https

Rating: 10.0/10 (1 vote cast)

Golden Dawn Representative attacks other MP

Posted on June 8, 2012 by Eva

A spokesman and member of parliament for Golden Dawn, a far far right (neo nazi) party in Greece assaulted another member of parliament on national TV. I wonder if the kind of people who voted for this kind of person will change how they vote after seeing this. Perhaps it will simply effect who Golden Dawn can go into government with. A lot will depend on how the party leadership handle this incident. I expect they will attempt to down play it, make excuses and shift the blame, however anything less than a full apology and firing the spokesman would be disgusting even if unsurprising.

Rating: 10.0/10 (1 vote cast)

Motivation, projects and what the fuck is laziness

Posted on June 8, 2012 by Eva

I have a technical book that will help me in my career sitting on my book shelf gathering dust. I was able to get through a similar book before, one chapter a day but for some reason I find this book far more intimidating.
I imagine that once I actually pick it up for the second time it will be a lot easier given I know more about the subject than I did when I first picked it up, but I’m lazy.

I don’t know what laziness is but for some reason that’s what I am. I feel lazy, meaning that instead of doing what I need to do to advance my career and also something I want to do because to me this subject is fascinating, I’m going to spend an evening starting a new blog, talking with friends and watching an episode of Hit & Miss (I’ll talk about this later).

My flatmate has a theory, he says that laziness is an expression of fear. I suppose it’s similar to my deer in headlights reaction when I’m put on the spot, my first reaction is to freeze up and try to hide, behind a computer screen if possible.

This isn’t working though, I’m going to have to deal with this, because I’m itching to make a difference, I want to influence this world and what I can do when I’m hiding just isn’t enough. I expect more from myself, always, but I always like to keep the distance in check.

I’ll admit I’m doing quite well though, only because I’m at the beginning of one of the most momentous changes in my life. Confirming my gender.

OK I’m going to finish this post off with an explanation of the profanity in the title. I’m a bit of pedant when it comes to language, someone describing a depressed person as lazy drives me up the wall. It makes me want to start a paramilitary force for the accurate naming of things. The profanity helps you, the reader understand that this is an emotionally charged issue for me, the author.

You’re welcome

Rating: 10.0/10 (1 vote cast)

The war on the internet

Posted on March 27, 2012 by Eva

It seems like a large number of western countries have plans in the works to heavily reduce Internet freedom.

Recently there was the US with SOPA and ProtectIP. ACTA is on it’s way and it’s international, The Netherlands have blocked the pirate bay as well as sites that attempt to offer proxies, those are just some of the countries I’ve seen recently introducing Internet filtering. And it’s scary, but what really worries me is when in a country like France, this kind of thing comes out : http://lubbockonline.com/..terror-websites..

For those who haven’t seen or read 1984, now is the time to understand what thought crime means, because what is being suggested is making the very act of reading a crime.

If for some reason you do not see the danger in this law, consider the following, how do you know if a website is extremist until you read it? What if I’m writing a book about terrorism, can I get a license to view websites that may turn me into an extremist? Perhaps I can go through an education program that will teach me how to resist the lure of murdering children due to differences between their faith and mine?

Ask yourself another question: why is this law being brought in? Is it being brought in because of the recent murders by an Islamic Extremist? Did he spend a lot of time on the Internet or was it his trips to Pakistan that are more responsible for his education in extremism? Did Sarkozy consider banning trips to Pakistan or was Internet censorship his first thought?

Regardless of how it took for him to react like this, it didn’t come from nowhere, like many politicians in our western world, the Internet is an open flank for government. If the Arab spring has taught these powerful people anything it’s not just that the Internet is a powerful political tool but that it’s a powerful unifying force for an oppressed population. Like the congressmen who proposed SOPA he realizes now the power of the Internet, and wishes to ensure it doesn’t continue to challenge his authority.

I’ll leave you with this: http://betanews.com/…not-democracy-but-populism… Sarkozy claims that without Internet regulation by a government is Anarchy, a situation where ideas that appeal to a large number of people grow in power. To him this is a horrible and dangerous thought that requires legal prevention.

Rating: 7.7/10 (3 votes cast)

Sean Sherlock wants the dogs called off?

Posted on March 1, 2012 by Eva

Yesterday SOPA Ireland, a law the power for judges to force Internet Service Providers to block websites took effect. The law follows the example of many other countries who similarly block websites for various reasons including child porn, phishing and copyright infringement. Inevitably these black lists are kept secret and when they are leaked to the public they inevitably contain web sites with perceivable legitimate reason for being on the blacklist.

So while this is about copyright today, it reduces our internet freedom, future generations of Irish men and women will likely never come to expect the same freedom on the internet that we enjoy today.

Today Sean Sherlock, the minister responsible for it’s introduction called for those opposed to copyright law to call off the dogs. I’m opposed to copyright law and the most I’ve ever done in relation to Irish SOPA is go to a protest. Maybe he could clarify what he meant?

He mentioned that people shouldn’t refer to this as the Irish version of SOPA, I don’t understand his point of view, SOPA = Stop Online Piracy Act, the Irish law is designed to stop online piracy by blocking websites, so is SOPA. What exactly makes these laws different except their wording?

Rating: 9.7/10 (3 votes cast)

Improving SSL on IIS6

Posted on December 19, 2011 by Eva

IIS 6 by default uses MD5 hashing and low security encryption.
if you put the following into a reg file and run it it should remove some of the less secure cipher suites used by IIS.

Update 1 : Go to http://support.microsoft.com/kb/948963 to enable AES 256 (requires restart)

Update 2: If you install the above update or turn on FIPs compliance in your local security policy this will make you vulnerable to the beast attack.

Rating: 9.5/10 (2 votes cast)

I don’t believe

Posted on November 7, 2011 by Eva

Personally I’m a big fan of the Santa Clause myth. At an early age it’s hard to imagine that your parents can be wrong let alone intentionally mislead you. Learning the fact that Santa Clause doesn’t exist is an important part of developing an awareness that authority figures lie and their lies are almost always self serving.

The loss of Santa Clause is important because it destroys magical thinking. You may find it cute that when your child wishes upon a star, but you don’t want them to be vulnerable to manipulation.

Rating: 9.6/10 (5 votes cast)

Power attracts the already corrupt

Posted on November 6, 2011 by Eva

I’m not an expert in economics, I’ve been trying to understand the situation as much as I can in terms of things that I already know.

I watched a documentary on Enron, a company that had a culture of risk taking, that promoted those who were driven to make money, thus who had the hunger for more power, those who knew how to play the game.

What I took away from that film is that the problem isn’t an economic system, it’s a culture that gives power to those who desire power

Rating: 8.5/10 (2 votes cast)