net start "VMWare Agent Service"
net start "VMWare Authorization Service"
net start "VMWare DHCP Service"
net start "VMWare NAT Service"
"C:\Program Files (x86)\VMware\VMware Workstation\vmrun" start "C:\VirtualMachines\sqlserver.vmx"
"C:\Program Files (x86)\VMware\VMware Workstation\vmrun" start "C:\VirtualMachines\sharepoint.vmx"

Save the above file as a batch file and store in a directory accessible by PATH.

For stopping, reverse the commands, use “suspend” instead of “start”.
]]> http://vasya10.wordpress.com/2012/01/20/start-sharepoint-virtual-machines-via-command-line/feed/ 0 vasya10 http://vasya10.wordpress.com/2012/01/11/groovy-olc-2-strand-puzzle-1914/ http://vasya10.wordpress.com/2012/01/11/groovy-olc-2-strand-puzzle-1914/#comments Wed, 11 Jan 2012 01:08:17 +0000 vasya10 http://vasya10.wordpress.com/?p=261 ]]> I think it was the year 1986 when I participated in an interschool programming competition in Chennai. I wasn’t very good at math, but I did tag along with a couple of my friends who were darn good at it. There were three problems given that we have to solve using GW-BASIC in about 30 minutes. There were probably 100 participants. I did get a peep into a few poor guys inside the computer room who were typing our solutions into the computer blindly and see if they worked. I just remember only one of the problems. I didn’t get the correct solution, but one of my friends – Sowmy Srinivasan, did and win the competition.

Years later, I learned that this was a very interesting puzzle tackled by the genius Srinivasa Ramanujan. In the year 1914, PC Mahalanobis, a Kings college student in England, got hold of a puzzle from the Strand magazine.

“In a given street of houses with consecutive numbers between 50 and 500, find the house number, for which, the sum of numbers on the left is equal to the sum of numbers on the right”

In other words, given ‘n’ number of houses, find the house number ‘m’ that would sum(1..m-1) = sum(m+1..n).

Mahalanobis solved the problem by trial and error and went to Ramanujan to show it. Ramanujan was cooking some vegetables in the pan. The instant he heard the problem, Ramanujan, while still frying his veggies, not only gave the answer to the puzzle, but also provided the generic formula that would give infinite number of solutions. The problem belongs to the area called Continuous Fractions and is very fascinating. The problem can be reduced to x*x – Dy*y = ±1 and has been tackled by ~10th century Indian mathematician Brahmaguputa using a method called chakravala. I will probably write up on the “chakravala” method later. The Ramanujan anecdote is documented in several places on the net and a simple search will provide more interesting details.

So I was working on something in Groovy, I suddenly got reminded of this problem and was thinking if this can be solved as a Groovy OLC. I am going to use the brute-force method rather than solving via the quadratic equation, because it nicely demostrates the List capabilities of Groovy. It does run slow for large numbers, but speed or optimization is not the point here.

So here is Groovy OLC #2 – Ramanujan’s Continuous Fractions (via brute force method)

public test_ramanujan_continous_fraction() {
(1..100).each { n, houses = 1..n -> houses.eachWithIndex {m, i -> if (houses.subList(0,i).sum() == houses.subList(i+1,houses.size()).sum()) println "house# = $m, total_houses = $n" } }
}

Result:
house# = 1, total_houses = 1
house# = 6, total_houses = 8
house# = 35, total_houses = 49

Ignore the first one, as I am not bothered about boundary conditions, though its technically correct.

[6,8] = sum(1..5) = sum(7..8) = 15
[35,49] = sum(1..34) = sum(36..49) = 595

Lets retest it with the actual problem from the Strand magazine:

public test_ramanujan_continous_fraction() {
(50..500).each { n, houses = 1..n -> houses.eachWithIndex {m, i -> if (houses.subList(0,i).sum() == houses.subList(i+1,houses.size()).sum()) println "house# = $m, total_houses = $n" } }
}

Result:
house# = 204, total_houses = 288

This is the actual solution that Ramunjan gave instantly to the Strand’s puzzle.

From Groovy’s point of view, the most interesting thing is the second argument in the first closure – “houses” – is actually a variable defintion which is a function of the first argument, ie houses = f(n) !!!

In Java, you dont commonly see beginning separate braces in a new line, although its okay for the ending braces. While in .NET, its a norm to have curly braces in separate lines. Inline code is also very rare in Java and .NET, except for get and set, but in Groovy/JavaScript, it seems to be a very common occurrence. Some of these conventions evolve as a tradition and some right from the invention. In Pascal, structured code was considered art. In Python block indentations are obligatory. Code format conventions, just like naming conventions are very subjective and sometimes personal and if a team cannot agree on one, its a rough road. Although some modern IDE-s have smartened up and present the personalized formatted code to the developer, while the source control would store a standard version. Some of the preferences are driven by language features too.

Closures, especially compact ones, are very intriguing when presented in a single line and some times very challenging. Solutions to puzzles and problems when rewritten in a single line look very catchy. Those who have coded in Prolog would perhaps realize what it means to say that a piece of code is work of art. You can do a regular findBy logic in Prolog, but doing it “Prolog”-way via predicates is certainly challenging.

So here is a One-Liner-Closure (OLC) that calculates the prime numbers > 2.

Given: a list of natural numbers
Find: all the prime numbers

void find_prime_numbers() {
def list = 1..100
println list.findAll { x -> (2..Math.sqrt(x)).every { x % it != 0 } }
}

Result:
[3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97]

The biggest advantage of SharePoint is probably the lack of useful official documentation. Thanks to it, numerous blogs and forums propose solutions and answers to more numerous problems. Some of them work, several don’t, as problem domain is always vastly huge when compared to the answer domain. For example, a simple combination of misbehaving USB driver + a command window open in non-Admin mode + an IIS session stored in-proc + neighbor’s dog’s wheezing bark + Saturn in conjunction with Mars in Aries constellation – all these could combinedly error an API out in SharePoint. And obviously no forum, let alone technical support, would be able to provide an answer to that. The programmer has to use the final weapon – perseverance.

So I was trying to use the ClientContext to get some data out of my sharepoint site, which is in a VM. Created a simple console project and the following code to get data from a site:

public Web GetRootWeb() {
Web rootWeb;
Uri uri = new Uri("http://sp:2010");
using (ClientContext clientContext = new ClientContext(uri) {
clientContext.Credentials = CredentialCache.DefaultNetworkCredentials;
clientContext.Load(clientContext.Web);
clientContext.ExecuteQuery();
rootWeb = clientContext.Web;
}
return rootWeb;
}

No matter what variations I tried – credentials, permissions, I always got the following error: “Cannot contact site at specified URL http://sp:2010″. Luckily I had another web application in the same server. And that worked. So something must be wrong with this sharepoint site. This site was a migration from 2007, so first I thought it could be a possibility.

Step 1: Inspect the closed-source code
The ClientRequestException was being thrown from the ClientContext.EnsureFormDigest() method. Having worked with open source for several years gives that irritable itch of peeping into others code. So I fired up .Net Reflector and inspected the method:

The first thing the method does is get an asmx file – its actually the _vti_bin/sites.asmx as the following method shows:

So does this service work against a soap query? Also notice the 2nd and 3rd highlights. Both conditions (actually 3) throw the same exception and error message. So which one failed here? Was a non 200 http code returned? Was the responseContentType not text/xml? Or was m_formDigestInfo null? Pretty much written by a programmer who does not know the fundamentals of debugging. And thats probably why Microsoft’s code is closed, so strangers like me cant code review it. As a best practice, error conditions and error messages should have a monogamous relationship or else a dna test is required to find out who is whose child.

Step 2: Rinse with a different agent – Use “Soap” UI
I fire up soap-UI 4.0 and add the wsdl. Since the site is behind NTLM Windows authentication, a challenge request pops up, I enter the credentials for the site (in this case the farmadmin account) and nope – it does not work. I get a 302 Found error and thats followed by a 401 Unauthorized in the same trace. Well 401 is not what I got before. I try the same with the second web app but that works fine again.

Step 3: Burp it out
After turning on both the http log and error log in the soapUI, I conclude that the credentials may not be passed correctly. After reading this thread, I fire up Burp Suite, goto Options tab, check the “do www authenticate” option, add the servers and the user name/password/domain combination, turn the interceptor off. Well, the 401 is not there anymore, because the BurpSuite makes sure that the header contains the username/password. But the 302 Found still happens.

Step 4: Always exchange dirty notes for good notes in the bank
Obviously IIS is doing some thing here. I replaced the sp:2010′s web.config with the second webapp’s web.config in-toto, recycle app pool and try to hit with soapUI. Voila, Im getting a successful connection now. So some line in the web.config is the culprit. Next I reverted the web.config and started aping section-by-section from the working web.config and I finally come to this line:

<!-- <sessionState mode="SQLServer" timeout="60" allowCustomSqlDatabase="true" sqlConnectionString="Data Source=db;Initial Catalog=SessionStateService_f84b37c3424f46afa09cdacd278a95fa;Integrated Security=True;Enlist=False;Connect Timeout=15" /> --?
<sessionState mode="InProc" cookieless="AutoDetect" timeout="20" />

This section is from the trouble-some site’s web.config. I uncomment the first line, comment the InProc mode line, recycle app pool and whoa! I get a successful connection. So the sessionState set to InProc mode is somehow influencing the soap query which is what the EnsureFormDigest uses and throws an ambiguous ClientRequestException.

I don’t know why a seemingly unrelated sessionState causes an issue with soap query when trying to connect via console application. Thats probably a reflection for another Christmas day. Atleast it works now from Console. I had to make a slight modification while trying to do the same from a web application. From console, the DefaultNetworkCredentials works fine, but from a web app, you have to use the actual site credentials to connect to the site.

The Title is a tribute to Perry Mason novels.

java.lang.NoClassDefFoundError: weblogic/utils/wrapper/WrapperImpl
…. (application related methods)…

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

But the same use case worked fine within the web environment. Turns out that the missing jar was com.bea.core.utils.wrapper-1.3.0.0.jar. Add this to your test classes or if you are using Maven – add it to pom.xml in test scope:

<dependency>
<groupId>oracle.weblogic.modules</groupId>
<artifactId>com.bea.core.utils.wrapper</artifactId>
<version>1.3.0.0</version>
<scope>test</scope>
</dependency>

One of the greatest assets of Groovy syntax is how easy it makes to manipulate lists and maps. In fact, the second most clumsy part of Java syntax and implementation are the Collections. The Calendar takes the honors. Collections are functional, but very cumbersome and laborious to implement, ie when compared to Groovy. Even if one is hesitant to learn functional programming or succumbing to the Obsessive Closure Designer syndrome, it is illustrative to know how easy it is to manipulate collections in Groovy.

def mapOfListValues = [a:[1,2], b:[3,4], c:[5,6], d:[7,8]]

So find the key to which a given number belongs to and return a default value if not found.

def findKey = { value -> (mapOfListValues.find { value in it.value }?.key)?:'Not Found' }
assert findKey(3) == 'b'
assert findKey(10) == 'Not Found'

My friend says that I should also put equivalent Java code to lighten up readers’ minds . So here goes …

HashMap<String, List<Integer>> mapOfListValues = new HashMap<String,List<Integer>>() {
{
put("a", Arrays.asList(1,2));
put("b", Arrays.asList(3,4));
put("c", Arrays.asList(5,6));
put("d", Arrays.asList(7,8));
}
};

public String findKey(...) {
blah blah, iterator, find, check if rest of world is not null,  catch exceptions and then a lot of code goes here
}

Yawn… you get the point..

One of the nicer features of Groovy is the ability to return multiple return values from a method via an implicit List and mapping them automatically to variables in the same order.

While iterator methods are very useful in Groovy, you have access only to the current “it” object within an iterator closure. There are times you may need access to a previous and next values too, just like in a Doubly LinkedList. Here is a smart closure that demonstrates meta programming technique (ie dynamically adding a method to java.util.List), multi-return values and iterators.

//create a [prev,current,next] list triplet from an array
List.metaClass.triplet = { i -> delegate.size() <= 0 ? [null,null,null] : i == 0 ? [null, delegate[i], (delegate.size()>1) ? delegate[i+1] : null] : (i==delegate.size()) ? [(i<1) ? null : delegate[i-1]] : [delegate[i-1], delegate[i], delegate[i+1]] }

Usage:

def list = [1,2,3,4,5,6,7,8,9,10]
for (int i=0; i<list.size(); i++) {
def (prev, current, next) = list.triplet(i)
}

This will print the following output:

[null,1,2]
[1,2,3]
...
[9,10,null]

The http-builder library is pretty useful to create http requests and responses in groovy. I was initially a bit surprised that groovy does not have this functionality built-in and had to import a separate jar.

Anyway I was trying to read a json response from the http-builder project website and the recommended sample did not work for some reason. This post provides a simple alternative.

Original (non-working) code

import groovyx.net.http.*
import static groovyx.net.http.ContentType.*
import static groovyx.net.http.Method.*

def url = "http://www.url.com"
def path = "/path/resource"
def outputFile = "c:/temp/result.json"
def params = [one:1, two:2]

def httpBuilder = new HttpBuilder(url)
httpBuilder.request(GET, JSON) {
uri.path = path
uri.query = params //parameters passed to the url -> ?one=1&two=2

headers.'User-Agent' = 'Mozilla/5.0 Ubuntu/8.10 Firefox/3.0.4'
response.success = { resp, json ->
println resp.statusLine
outputFile.withWriter('UTF-8') { out -> out.println json }
}
}

The above code reads the json file, but if you open the json file in firefox with the JSONView plugin, you may get a “json is invalid” error.

Solution:

The updated solution is to read the response as a text and apply ‘application/json’ content type.

 def httpBuilder = new HttpBuilder(url)
 httpBuilder.request(GET, TEXT) {
  uri.path = path
  uri.query = params

  headers.'User-Agent' = 'Mozilla/5.0 Ubuntu/8.10 Firefox/3.0.4'
  headers.Accept = 'application/json'
  response.success = { resp, json ->
   println resp.statusLine
   outputFile.withWriter('UTF-8') { out -> out.println json.text }
  }
}

With 1.8.3 release and built-in json support, this seems to be even easier now, though does not seem to be much fine-tuning of response success and failures.

def params = [one:1, two:2, three:3]
def paramString = query.collect { "$it.key=$it.value" }.join("&")
def url = site + path + "?" + paramString
def jsonResult = new URL(url).text

This one is not exactly a SharePoint quirk, but a SharePoint app led me to it. Which means its an IIS quirk. Or a .Net quirk. Or a Service Pack quirk. Or a Windows Server 2008 quirk. Or a Microsoft quirk. Or whatever…

My co-worker had written a form-based-authentication login page for a SharePoint application. The users are created in Active Directory and the usual frills of user account – login, security questions, change password, reset password et al.

The app worked fine in dev environment, was working fine in Stage but one fine Monday, broke in Stage. The user was created in the AD, but was set in a disabled state and returned an error back to the application. It was working in Dev fine though. The web.config, wsp solution, content databases are all exactly same between dev and stage. The only difference was the stage environment had Windows Server 2008 R2 SP1, while the Dev was not. We decided to backout SP1 from stage and lo, it started working again. Could it be the sheer gravitons of SP1 cause to distort the user creation in the Active Directory?

Since SP1 was causing the problem, we got the dev environment to the same SP1 patch level as stage. And yes, the user creation broke. Repeatable bugs are like flights crashing without casualties. Yeah, it crashed, but no one died. I started looking at the ULS and noticed there were no log statements. Coming from the Java world, I rely more on debug statements rather than run-time debugging. A friend of mine reminds me that in the .Net world programmers first put the solution in Debug mode, then start writing the code. But here the log statements were there in the code, yet were not written.

You can spot the issue here:

public static class ExceptionExtension {
public static string Unroll(this Exception ex) {
if (ex == null) return "";
string exStr = ex.Message + "\n" + ex.StackTrace;

if (ex.InnerException != null) {
exStr += "\n" + ex.InnerException.Unroll();
}

return exStr;
}

Test Usage:
PortalLog.LogString("ERROR: {0} || [{1}]", errorMessage, ExceptionExtension.Unroll(ex));

The above code wont work, because the correct usage of the extension is ex.Unroll(), not by a static method call. Once I fixed this, I could see the actual errors in ULS.

Here is how the account was being created usig LDAP:

entry = new DirectoryEntry(ldapPath, ldapUser, ldapPassword, AuthenticationTypes.Secure);
//create new user object and write into AD
user = entry.Children.Add("CN=" + samAccountName, "user");
user.Properties["userprincipalname"].Add(emailAddress);
user.Properties["samaccountname"].Add(samAccountName);
//...
user.CommitChanges();
user.Invoke("SetPassword", password);
user.Invoke("Put", new object[] { "userAccountControl", "512" });

System.DirectoryServices threw exception with an error code 0×80070005 (ACCESS_DENIED) while invoking SetPassword. I double checked that the ldapUser has identical permissions in dev and stage. How can the user be created but still access denied? More importantly, WHO is being denied? Certainly not the ldapUser.

I added the debug for WindowsIdentity.GetCurrent().GetName(). And it returned NT AUTHORITY\IUSR. In IIS, the anonymous access is enabled and in the web.config the identity tag is

<identity impersonator="true"/>

Obviously IUSR does not have permission to set the password. But why was it working in dev, before SP1? When I tried in another environment without SP1, the current Windows Identity was the same as the Application Pool user for that SharePoint Web Application and not the IUSR. Google-o! Er.. I mean, Bing-o!

Back to dev with SP1, I set the impersonator to false, and now I saw the current identity as the Application Pool user! W2k8 R2 without SP1 was not respecting the identity impersonator flag. Obviously without SP1, the IUSR anonymous user was having access to create an user in active directory, while SP1 prevented it. The impersonation is not required when you are creating user using the ldap service user and password, who already have access.

To me it looks like a security hole (pre-SP1). I could not find any online documentation or release notes mentioning about this. If you know of such documentation, please drop a comment!

Is there something called “Over-architecting a solution”? Or do some people get carried away too much by Object Oriented Concepts that a simple if-else condition is enshrined into a dazzling hierarchy of classes, interfaces and er.. content types? SharePoint’s answer to creating a class/type or pseudo-interface is the Content Type. While a powerful feature, there are times even the Microsoft Team gets confused what to do with it, unconsciously bury a feature and consciously let it go undocumented.

There was a requirement to show different banners with different styles based on items in the “Banner List”. The different banner types were represented as “Content Types” and so the end user will create Banner Type 1, Banner Type 2 etc; set it to active and it would appear on the web page. The solution was designed in such a way that a calculated column in that list was populated based on the content type of the banner. That is, the calculated column had a formula in the BannerText sitecolumn:

=IF [Content Type] == "Banner Type 1" Then BannerText = "Red Banner" ELSE IF [Content Type] == "Banner Type 2" Then BannerText = "Blue Banner" ELSE IF...

The site column was relying on the Content Type of the list, that will be known at runtime. And that was working perfectly fine. In SharePoint 2007, that is.

Over to 2010. When I was trying to add a new item in that list, the calculated column Text would populate with a value “#NAME?” indicating that there is something wrong with the formula. Going back the site column, initially said the formula is fine, but when I tried to update it, errored out saying invalid column. Upon inspection, I see that Content Type is missing in the “Insert Columns” list.

First I thought my migration was wrong. Looked back at the SharePoint 2007 implementation, the “Content Type” column is there (see above image from MOSS 2007)! So Microsoft Team couldn’t handle the power the Content Types and removed it, I guess. But why was it removed? (see image below, from SharePoint 2010)

The _layouts/fldnew.aspx page inherits from Microsoft.SharePoint.ApplicationPages.BasicFieldEditPage. The method that creates the Insert Columns is WriteFieldNamesForFormulaColumnPicker(), so I loaded the .Net Reflector and inspected the code.

protected internal static void WriteFieldNamesForFormulaColumnPicker(TextWriter output, SPFieldCollection fields, string currentDisplayName)
 {
 SortedList list = new SortedList(new Comparer(fields.Web.Locale), fields.Count);
 foreach (SPField field in fields)
 {
 if ((((!(field is SPFieldText) && !(field is SPFieldCalculated)) && (!(field is SPFieldNumber) && !(field is SPFieldDateTime))) && (!(field is SPFieldChoice) || (field is SPFieldWorkflowStatus))) && (!(field is SPFieldBoolean) || (field is SPFieldAllDayEvent)))
 {
 continue;
 }
 if (((field.InternalName != "_UIVersionString") && !field.Hidden) && (field.Title != currentDisplayName))
 {
 list.Add(field.Title + "_" + field.InternalName, field);
 }
 }
 bool flag = true;
 foreach (SPField field2 in list.Values)
 {
 string title = field2.Title;
 SPHttpUtility.NoEncode("<option value=\"[", output);
 SPHttpUtility.HtmlEncode(title, output);
 SPHttpUtility.NoEncode("]\"", output);
 if (flag)
 {
 SPHttpUtility.NoEncode(" selected=\"selected\"", output);
 flag = false;
 }
 SPHttpUtility.NoEncode(">", output);
 SPHttpUtility.HtmlEncode((title.Length > 20) ? (title.Substring(0, 20) + "...") : title, output);
 SPHttpUtility.NoEncode("</option>", output);
 }
 }

So if the field is only one of text, calculated, number, datetime, choice, workflowstatus, boolean it is available for formulas. So what type is the Content Type column anyway? Using Powershell, we can determine that –

PS C:\Users\spfarmadmin> $w.Fields["Content Type"]

EnableLookup                : False
FieldRenderingControl       : Microsoft.SharePoint.WebControls.ComputedField
FieldRenderingMobileControl : Microsoft.SharePoint.MobileControls.SPMobileComputedField
FieldTypeDefinition         : Microsoft.SharePoint.SPFieldTypeDefinition
TypeDisplayName             : Computed

So since the field is Computed type, it does not appear in the insert column list. The same code in SharePoint 2007 didn’t have this restriction. So there is no way to find content type from within a site column. So I had to redesign the list now. Instead of multiple content types, I just added a new dropdown column which contain a list of values which was exact same as the content type. That way the functionality still worked without changing the underlying custom code. So a list with several content types and calculated columns to determine a text could be replaced with a simple dropdown list! Instead of multiple content types and formulas.

But a fundamental feature was removed. May be the next release of SharePoint 2014 will not have content databases?