Bug 2613

JSONParser and JSONPackage do not handle null values 20 November, 2020

Kevin Ridley
11 April, 2019
Product: PowerBuilder Category: PowerScript
Version: 2017 R3 Build:
Classification: Publishing: Public
Priority: P3
Status: Verifying Reason:
Ken Guo @Appeon 20 November, 2020
#15
Hi Kevin,

I would like to check with you if the response provided has been helpful enough
to resolve your case. If so, can we proceed to close the ticket?
Otherwise, do let us know if we can provide you with any further assistance and
keep in mind that you are always welcome to open another ticket in case you
need it.

Regards,
Ken
Ken Guo @Appeon 07 May, 2019
#14
Hi Kevin,

1. As for the Null problem, I have submitted it to the developer team and they have included it in the discussion planning. The developer team will make a decision of whether the GetValue*** function can get Null value after comprehensive analysis. 
2. PB 2017 R3 is a TLS version. In order to keep this version as stable as possible, we will only fix its bugs and won’t extend the functionality. So the new features in PB 2019 won’t be synchronized with PB 2017 R3.

Appreciate your understanding.

Regards,
Ken
Kevin Ridley 06 May, 2019
#13
Thanks Ken.  These are all good enhancements, but is mostly a workaround for the problem, not an actual fix.  I apologize if this sounds funny, but in addition to ReturnsNullWhenError, why can't we have a ReturnsNullWhenNull?  Having a Null value in JSON is NOT an error, it's a very valid condition.  As I mentioned, this prevents me from recommending PB as an Enterprise level solution for working with JSON.  Also, we are being told that PB2017 R3 is a LTS solution, so I can't imagine why these changes wouldn't be synch'd with PB2017 R3.  I appreciate your response though.
Kevin Ridley 06 May, 2019
#12
Thanks Ken.  These are all good enhancements, but is mostly a workaround for the problem, not an actual fix.  I apologize if this sounds funny, but in addition to ReturnsNullWhenError, why can't we have a ReturnsNullWhenNull?  Having a Null value in JSON is NOT an error, it's a very valid condition.  As I mentioned, this prevents me from recommending PB as an Enterprise level solution for working with JSON.  Also, we are being told that PB2017 R3 is a LTS solution, so I can't imagine why these changes wouldn't be synch'd with PB2017 R3.  I appreciate your response though.
Ken Guo @Appeon 05 May, 2019
#11
Hi Kevin,

There are some improvements in PB 2019 about JsonParse and JsonPackage. Please refer to the below for more details:
https://www.appeon.com/support/documents/appeon_online_help/pb2019/whats_new/ch01s05.html
https://www.appeon.com/support/documents/appeon_online_help/pb2019/whats_new/ch01s06.html 

However, sorry that we still don’t have a plan to synchronize the improvements to PB 2017 R3 for the time being.


Regards,
Ken
Kevin Ridley 03 May, 2019
#10
Is there any update on these issues?  These are very significant bugs.  If I am going to recommend PB as an enterprise capable solution for REST web services and it can't process JSON with nulls or get a True/False value, then it's not going to be a viable solution for a production worthy environment.  Please advise.
Chris Pollach @Appeon 12 April, 2019
#9
 No problem now as they will see your last comment!  ;-)
Kevin Ridley 12 April, 2019
#8
(In reply to Chris Pollach from comment #7)
Hi Kevin;

  Yes as a matter of fact, when I was updating my Integrated Framework and
example OrderEntry application way back when JSON processing was first being
introduced, I found these issues around handling NULLS in the JSON object
classes. I then informed our Engineering folks. I think that helped lead
Engineering to address this issue in PB2019.

  However, your question as to needing this support in PB2017R3 is 100% "on
the money" for the LTS version and thus, the $64M question!

  I will now transfer this ticket over to the main Support/Engineering team
to advise us on the new JSON support aspects in PB2019 and when we might be
able to get these features in a PB2017R3 maintenance release.

Regards ... Chris
Thanks Chris. Please be sure to make them aware that we need more than what's currently in 2019 to make this robust and enterprise worthy for processing JSON. We really need everything I mentioned and especially for the JSONPackage object.
Chris Pollach @Appeon 12 April, 2019
#7
Hi Kevin;

  Yes as a matter of fact, when I was updating my Integrated Framework and example OrderEntry application way back when JSON processing was first being introduced, I found these issues around handling NULLS in the JSON object classes. I then informed our Engineering folks. I think that helped lead Engineering to address this issue in PB2019.

  However, your question as to needing this support in PB2017R3 is 100% "on the money" for the LTS version and thus, the $64M question!

  I will now transfer this ticket over to the main Support/Engineering team to advise us on the new JSON support aspects in PB2019 and when we might be able to get these features in a PB2017R3 maintenance release.

Regards ... Chris
Kevin Ridley 12 April, 2019
#6
(In reply to Chris Pollach from comment #5)
Hi Kevin;

  The JSONParser object has also been updated in PB2019 as well ... for
example:

1) Set/GetValue<DataType>() methods to properly handle data natively (vs all
"string" in PB2017R3). In your case Get/SetValueBollean() for example.
2) New GetItemType() method that would now return "JsonNullItem!".
3) New GetValueToDataWindow () method for DC & DS controls.
4) New property "ReturnsNullWhenError" was added to trap NULL Error
exceptions.
5) Various fixes to the JSONParser and JSONGenerator objects
etc

Have you tried any of these features using the PB2019 Beta#1 or #2?

Regards ... Chris
Hi Chris, I haven't had time to try the 2019 Beta yet. Our company plans are to use 2017 R3 since it is LTS and we are spending months testing it for our conversion from 12.5, so we don't envision going to 2019 in the near future. That being said, we need these bugs fixed in 2017. Have you tried using the JSONPackage object? It's much more useful than the Parser. I'd love to see that object enhanced more instead of the Parser, or combine them into a more powerful parser. Either way, we need the nulls bug fixed, support for boolean in the JSONPackage.GetValue, and the datawindow should support boolean datatype for ImportJSON or convert it to a string so we can use it as a string in a dw.ImportJSON. Thanks.
Chris Pollach @Appeon 11 April, 2019
#5
Hi Kevin;

  The JSONParser object has also been updated in PB2019 as well ... for example:

1) Set/GetValue<DataType>() methods to properly handle data natively (vs all "string" in PB2017R3). In your case Get/SetValueBollean() for example.
2) New GetItemType() method that would now return "JsonNullItem!".
3) New GetValueToDataWindow () method for DC & DS controls.
4) New property "ReturnsNullWhenError" was added to trap NULL Error exceptions.
5) Various fixes to the JSONParser and JSONGenerator objects
etc

Have you tried any of these features using the PB2019 Beta#1 or #2?

Regards ... Chris
Kevin Ridley 11 April, 2019
#4
(In reply to Chris Pollach from comment #3)
Hi Kevin;

   FYI: This is a short-coming in PB2017R3 MR01 (build (1880) but is
addressed in PB2019, as follows:

1) New "ContainsKey" command

    Allows you to verify if the "Key" exists before you ask the parser to go
after a NULL key / value.

2) New "GetItemType" command

    Allows you to check the JSON data type before issuing a GetItemXxxxx()
command

3) New "ReturnsNullWhenError" setting

    Allows you get a NULL back instead of throwing an exception

HTH

Regards ... Chris
Chris, Everything you mentioned above is for the JSONParser only. The JSONPackage already has a ContainsKey but still barfs on null values. The ContainsKey is valuable on the Parser too, but it's still a bug when the value is null. The last setting helps but still you won't know if you got a null because of an error or if the value was null. We really need full support for null values if we are really going to process JSON in production quality apps. We also need to handle boolean types in JSON with the JSONPackage object. Same thing with the Datawindow.ImportJSON, we need a boolean column type or at the very least convert boolean to string.
Chris Pollach @Appeon 11 April, 2019
#3
Hi Kevin;

   FYI: This is a short-coming in PB2017R3 MR01 (build (1880) but is addressed in PB2019, as follows:

1) New "ContainsKey" command

    Allows you to verify if the "Key" exists before you ask the parser to go after a NULL key / value.

2) New "GetItemType" command

    Allows you to check the JSON data type before issuing a GetItemXxxxx() command

3) New "ReturnsNullWhenError" setting

    Allows you get a NULL back instead of throwing an exception

HTH

Regards ... Chris
Kevin Ridley 11 April, 2019
#2
"Same thing if you try a JSONParser.GetValue for a boolean value."
should be
Same thing if you try a JSONPackage.GetValue for a boolean value.
Kevin Ridley 11 April, 2019
#1
test.json (0KB)

*Phenomenon: When using JSONParser and JSONPackage objects, when you try to JSONParser.GetItemString or JSONPackage.GetValue on a null value, it causes a RuntimeError.  Same thing if you try a JSONParser.GetValue for a boolean value.  In this case PB should convert it or any other not supported type to a string.  Currently (GetValue) throws an error saying The returned item of the GetValue function has incorrect data type.  Expected type: object, array or string.


*Reproduce Steps:
Use LoadString for the attached json and try to access a null or boolean value.

Remarks:
Both should be fixed to handle null values.  Support for boolean type should also be added.  It should also be added as a datawindow column type so that ImportJSON will work correctly.  At the very least, fix the nulls and convert boolean to string.
OS:
Windows 10
Platform:
All
Database Type:
N/A
Database Version:
N/A