wissel.net

Usability - Productivity - Business - The web - Singapore & Twins

Reader Fields - extreme Edition (update)


Reader fields are "haunting" me. Recently I was asked to provide a solution for a requirement to add a very large number of entries to a reader field. A number that firmly exceeds the storage capability of a reader field (still 32k after all these years). My first instinct was to look for alternatives like using a few group names. However the alleged business case is the exclusion of a few people from a large group (somehow: the whole department can read a document except the person who has birthday since it is a party-planning application). This was one of the cases where I was wishing for a new field-type: NoReaderField (and for that sake NoAuthorField).
Well there is always the next version. Until then I created a small class, that allows you to add, remove and query names in a "virtual" reader field. Instead of reading and writing into a Notes item the developers call a method in a class (addEntry(newName as String) or addEntries(newNames as Variant) (an Array)) to interact with the reader field. The "saveField" method then distributes the values onto one or more reader fields. I don't made it too scientific and counted bytes, but rather count number of names. Splitting the reader names across multipe fields requires adjustments to your views. Something like Reader1:Reader2:Reader3:....:Readerx. We haven't tested the impact of that on performance. Once I have data on that I'll provide an update.

Update: Turns out the class was a bad idea. There is a hard stop for the total size of all combined Reader fields in a document. So the alternative is to use an approach similar to the NoReader field. I consulted with the master and discuss alternatives.

Have a look at the code yourself:
 
Option Public Option Declare'/**' * ' * FieldAccess is a universal class to manage large amount of values put into text fields ' * (text, names, readers, authors etc. ' * it manages these large amount by splitting reader names into multiple fields to overcome ' * the 32k size limit. New fields are created as needed. ' * Good for 62500 entries ' * ' * v0.1 2009-03-20 : notessensei@sg.ibm.com '**/'// Important: This class doesn't contain production quality error handling, you need to add '// that for production use. Use OpenLog for best results!PublicClass FieldAccess Private AccessFieldName AsString'The base name for the Reader fieldPrivate maxFieldEntries AsInteger'The maximum number of entries per reader fieldPrivate maxNumOfFields AsInteger'Limit the number of fields, to avoid mising data in views Private maxTotal AsInteger'The maximum number of names in totalPrivate curTotal AsInteger'The current number of entriesPrivate curFieldMembers ListAsString'The list of current Reader entriesPrivate fieldCount AsInteger'The current number of fieldsPrivate initialized AsBoolean'Has the list of names been initializedPrivate doc AsNotesDocument'The current documentPrivate fieldType AsInteger'Text, names, reader, authorSubnew(curDoc AsNotesDocument)SetMe.doc = curDoc Me.AccessFieldName ="Field"Me.maxFieldEntries = 500 Me.maxNumOfFields = 10 'Adjust as neededMe.initialized =FalseMe.fieldType = 0 'No special typeEndSub'Need to set the field NamePublicPropertySet fieldName AsStringIfMe.AccessFieldName <> fieldName ThenMe.AccessFieldName = fieldName Me.initialized =FalseEndIfEndPropertyPublicPropertyGet FieldName AsString fieldName =Me.AccessFieldName EndProperty'How many entries are in the names field right nowPublicPropertyGet count AsIntegerIfNotMe.initialized ThenMe.initializeFields EndIf count =Me.curTotal EndProperty'Add a entry to the value listPublicSub addEntry (entryToAdd AsString)IfNotMe.initialized ThenMe.initializeFields EndIfIfNotIselement(Me.curFieldMembers (entryToAdd ))ThenIfMe.curTotal =Me.maxFieldEntries *Me.maxNumOfFields ThenError 8000 'We throw an errorElseMe.curFieldMembers (entryToAdd )= entryToAdd Me.curTotal =Me.curTotal + 1 EndIfEndIf EndSub'Bulk adding of entries. Suitable to pull in entries from getItemValues
PublicSub addEntries (entriesToAdd AsVariant)Dim oneEntryToAdd AsStringIfNotIsarray(entriesToAdd )ThenError 8001 'We won't process something that's not an arrayEndIfForall curEntry In entriesToAdd oneEntryToAdd = curEntry CallMe.addEntry (oneEntryToAdd )EndForallEndSub'Entry to remove. Is removed from thefields if exist. If the entry'doesn't exist no error is raisedPublicSub removeEntry (entryToRemove AsString)IfNotMe.initialized ThenMe.initializeFields EndIfIfIselement(Me.curFieldMembers (entryToRemove ))ThenEraseMe.curFieldMembers (entryToRemove )Me.curTotal =Me.curTotal - 1 EndIfEndSub'Bulk removal of Entries. Suitable to pull in values from getItemValuesPublicSub removeEntries (entriesToRemove AsVariant)Dim oneEntryToRemove AsStringIfNotIsarray(entriesToRemove )ThenError 8001 'We won't process something that's not an arrayEndIfForall curEntry In entriesToRemove oneEntryToRemove = curEntry CallMe.removeEntry (oneEntryToRemove )EndForallEndSub'Checks the reader fields for the existance of and entryPublicFunction containsValue (valueToLookup AsString)AsIntegerIfNotMe.initialized ThenMe.initializeFields EndIf containsValue =Iselement(Me.curFieldMembers (valueToLookup ))EndFunction'Saves the changes in fields back to the documentPublicSub saveField Dim curFieldCount AsIntegerDim tmpCount AsIntegerDim tmpTotal AsIntegerDim tmpMax AsIntegerDim allEntries ()AsString'We save the names in chunks of maxFieldEntries'into Names fields and remove fields we don't need anymore curFieldCount = 1 tmpTotal = 0 tmpCount = 0 'Size the array to fit either all entries or the max# of entriesIfMe.maxFieldEntries <Me.curTotal ThenRedim allEntries (Me.maxFieldEntries )ElseRedim allEntries (Me.curTotal )EndIf'Now we go through the names and put them into field bucketsForall curEntry In Me.curFieldMembers If tmpCount >Ubound(allEntries )Then'Save the current fieldCallMe.SaveToField (allEntries ,curFieldCount )'Reset the array'//TODO: Check if this has an off-bye-one-error!IfMe.maxFieldEntries >(Me.curTotal - tmpTotal )ThenRedim allEntries (Me.curTotal - tmpTotal )ElseRedim allEntries (Me.maxFieldEntries )EndIf tmpCount = 0 curFieldCount = curFieldCount + 1 EndIf tmpTotal = tmpTotal + 1 allEntries (tmpCount )= curEntry 'Get to the next tmpCount = tmpCount +1 EndForall'We need to save the last batch tooCallMe.SaveToField (allEntries ,curFieldCount )'Remove field we might not need anymoreIf curFieldCount <Me.fieldCount ThenCallMe.RemoveFields (curFieldCount +1 ,Me.fieldCount )Me.fieldCount = curFieldCount EndIfEndSub'Saves one fieldPrivateSub SaveToField (curVals AsVariant, fieldCountOffset AsInteger)Dim fieldName AsStringDim newField AsNotesItem fieldName =Me.AccessFieldName +"_"+Trim(Cstr(fieldCountOffset ))If doc .HasItem (fieldName )ThenCall doc .RemoveItem (fieldName )EndIfSet newField =NewNotesItem(doc ,fieldName ,curVals ,Me.FieldType )EndSub'Removes fields we don't needPrivateSub removeFields (startCount AsInteger, endCount AsInteger)Dim fieldName AsStringDim newField AsNotesItemDim i AsIntegerFor i = startCount To endCount fieldName =Me.AccessFieldName +"_"+Trim(Cstr(i ))If doc .HasItem (fieldName )ThenCall doc .RemoveItem (fieldName )EndIfNextEndSub'Initialize the fieldsPrivateSub initializeFields Dim namesField AsNotesItemDim i AsIntegerDim curEntry AsStringDim curVals AsVariantIf doc .HasItem (Me.AccessFieldName )ThenMe.curTotal = 0 '//TODO:Full check for missing 1st field i = 1 DoWhileTrue curEntry =Me.AccessFieldName +"\_"+Trim(Cstr(i ))IfNot doc .HasItem (curEntry )Then i = i -1 'We overcounted one, so we step backExitDoEndIf curVals = doc .GetItemValue (curEntry )'Save the names into a list for processingForall member In curVals IfNotIselement(curFieldMembers (member ))Then curFieldMembers (member )= member Me.curTotal =Me.curTotal + 1 EndIfEndForall i = i + 1 LoopMe.fieldCount = i ElseMe.fieldCount = 0 Me.curTotal = 0 EndIfMe.initialized =TrueEndSubEndClass'/**' _' _ ReaderAccess is a universal class to manage large amount of reader names in a document'**/'// The only difference to FieldAccess is the FieldName and the FieldTypePublicClass ReaderAccess As FieldAccess SubNew(curDoc AsNotesDocument)'The base class runs first, so we only need to add the changesMe.AccessFieldName ="DocReaders"Me.fieldType =READERS'Set type to READEREndSubEndClass
This LotusScript was converted to HTML using the ls2html routine,
provided by Julian Robichaux at nsftools.com.

Posted by on 15 April 2009 | Comments (2) | categories: Show-N-Tell Thursday

Comments

  1. posted by Srinivas Kotaru on Wednesday 15 April 2009 AD:
    Wow. Great piece of code. Thanks a lot.
  2. posted by Kevin Pettitt on Wednesday 15 April 2009 AD:
    Nice one Stephan.

    I'm guessing Reader fields have to be "Summary" fields, but if not you could double the size of the field contents to 64K. No biggie since once you're building multiples on the fly the number probably isn't that important. If performance sucks though it might be worth investigating. Also, if that direction proves useful, you have to make sure the corresponding form doesn't have the Reader_1, etc. fields on it or the summary flag will be reapplied if saved in the UI (or at least it'll try).