Bug 5844

Build Automation with PowerBuilder Git Integration 14 January, 2021

Ankur Patel
06 January, 2021
Product: PowerBuilder Category: Other
Version: 2019 R2 Build: 2353
Classification: Enhancement Publishing: Public
Priority: P3
Status: Closed Reason: RECORDED AS ENHANCEMENT
Appeon Product 14 January, 2021
#7
Hi Ankur,  

 

Thank you for submitting the ticket.  

We now acknowledge this ticket to be an enhancement request and will record and track it internally in our enhancement management system. At the same time, we are closing the ticket here because this support system is for handling standard support services but not for maintaining enhancement requests.  

Please note that despite its "Closed" status, you are welcome to come back here and leave further comments or query the relevant development progress on this ticket.  

 

Regards,  

Amy Feng
Ken Guo @Appeon 11 January, 2021
#6
Hi Ankur,

Sorry that I don’t know when it will be available.
Every time when we want to release a new version, our development team and product team will decide which new functionalities can be implemented based on their importance, workload, risks, priority, etc. After the evaluation, they can decide what can be developed and when it can be implemented. 
We need to implement the decided new functionalities in priority with limited resources. Hope you can understand.

So for now, I can’t give you an exact date for this request including the pb190.exe with /refresh parameter request. 
However, I have submitted these two requests to the product team and development team again and also listed the advantages of the requests to them for consideration. 

Regards,
Ken
Ankur Patel 11 January, 2021
#5
Hello Ken,

Thanks for the update. I am also hoping that the new "sync_all" option will also work with "outofdate", so basically the following command will do almost the same thing that that IDE does when we refresh the target.
scc set target "D:\temp_svn\datawindow_json.pbt" "outofdate sync_all"
So this will only sync the changed, deleted, and newly added objects and it will only need to compile the changed or new objects and not everything else making the exe generation faster.

Can you also tell me when will this new option be available. If this is going to take time, is it possible that you guys can release the temporary workaround where we can call pb190.exe with /refresh parameter to refresh the PBLs in the meantime? 

Thanks
Ken Guo @Appeon 07 January, 2021
#4
Hi Ankur,

Currently when ORCAScript and ORCA API are using GIT & SVN, they only support scc connect offline mode, online is unsupported. So GIT/SVN can’t be connected, so they also can’t support the ‘Refresh’ action like that in PB IDE.

To support this functionality, I have submitted a new requirement to the product team and hope we could support command like below:

scc set target "D:\temp_svn\datawindow_json.pbt" "refresh_all sync_all"

Compare the newly added ‘sync_all’ parameter with the ‘importonly’ parameter, besides executing Import, ‘sync_all’ will also remove the redundant objects in pbl file.

At present, you can only delete all PBLs first and then execute ORCAScript. Of course, this will take time.

Regards,
Ken
Ankur Patel 07 January, 2021
#3
Hello Tom,

I know the behaviour is the same as the traditional SCC API interface. We were not using the traditional SCC API before. Before migrating to PB 2019 R2 we were directly working with PBL. We were directly checking out and checking in the PBLs. 

Now, because this functionality is not available, the only option we are left with is to delete the PBLs and create all the PBLs from scratch which takes up too much time. Since you guys have already developed this feature in PowerBuilder IDE, would it be that hard to add this option in ORCAscript as well. And meanwhile, if you can add a /refresh command line parameter in PB190.exe that would work as well. 

Also, Since PB190.exe already has this capability, Can you point me in some direction where I can prepare some script myself for the time being to do the same thing that IDE does when we refresh the workspace? Do you think we can do that using Orcascript or do we have to use ORCA API?
Tom Jiang @Appeon 07 January, 2021
#2
Hi Ankur,

It seems the behavior is the same when using the traditional SCC API interface. Did you successfully use the traditional SCC API interface with OrcaScript to remove objects from existing PBLs? If you did, please share you script for us to further analyze the issue.

If there is no such an option with the traditional SCC API interface either, I will pass it to our product team as an enhancement request.  

Regards,

Tom Jiang
Chris Pollach @Appeon 06 January, 2021
#1
Hi Ankur;

  Thank you for your insights into the build process and your enhancement suggestions. You are correct, once a PB App is interfacing to SCM software, you must use the ORCAScript utility to complete the first stage of the build process on another deployment machine vs the IDE's PC.

  I will now transfer this ticket over to the main Support / Engineering team for their review & feedback.

Regards ... Chris
Ankur Patel 06 January, 2021
There is no streamlined way to create EXE when using the new PowerBuilder Git integration. I am going to list down all the approaches that I have tried.

1. The official way to automate the building of exe is using orcascript as described in the following link.
https://docs.appeon.com/pb2019r2/pbug/apbs04.html

The suggested script for this is as following:
start session 
scc set connect property localprojpath "D:\temp_svn" 
;localprojpath must point to the parent directory of ws_objects folder
scc set connect property logfile  "D:\temp_svn\svnorca.log" 
scc set connect property logappend false
scc set connect property deletetempfiles "true"
scc connect offline
scc set target "D:\temp_svn\datawindow_json.pbt" "refresh_all importonly"
scc exclude liblist "D:\temp_svn\pbsoapclient190.pbd"
scc refresh target incremental
scc refresh target "3pass" 
build library "D:\temp_svn\datawindow_json.pbl" "" pbd 
build library "D:\temp_svn\appeon_workarounds.pbl" "" pbd
build library "D:\temp_svn\qa_frame.pbl" "" pbd
build executable "D:\temp_svn\datawindow_json.exe" "" "" "yyy"
scc close
end session

There are 2 issues with this approach
	1. When you do "scc set target "D:\temp_svn\datawindow_json.pbt" "refresh_all importonly"", it imports all the objects from ws_objects folder to the pbls. But it does not delete the objects that are there in the pbl but not there in the ws_objects folder anymore. This issue was already reported by Rene here: https://www.appeon.com/standardsupport/search/view?id=3083. That issue has been recorded as an enhancement, but in my humble opinion, this is a bug. You are easily able to do the same when we refresh the PowerBuilder IDE. I don't think it is that much of an effort to include the functionality to delete the objects from pbls that are not there in ws_objects.
	2. Because of the issue above, the only remaining solution is to delete all the pbls before running the above-mentioned orcascript. But the problem with that is, it takes up too much time. In our project where we have about 200 pbls, it takes up more than 2.5 to 3 hours to bootstrap all the pbls and then generate pbds and exe. While if I do the same thing manually using PowerBuilder IDE, I open the PowerBuilder, refresh the workspace which takes about 5 minutes and then I build the exe using IDE which takes up around 45 minutes. So the EXE, which could be built within 50 minutes, takes up more than 2.5 hours.
	
I would like to suggest a few potential fixes for this approach.
	1. Add functionality in orcascript so when we do scc set target "D:\temp_svn\datawindow_json.pbt" "refresh_all importonly", it should also delete objects which are no longer there in ws_objects. Even better would be: "scc set target "D:\temp_svn\datawindow_json.pbt" "outofdate importonly", so it would only import the changed objects and only delete the objects which are deleted. So this will become exactly like the IDE refresh.
	2. Other option, which I think might be easier for you to release early would be to add a commandline parameter in the PowerBuilder IDE itself to refresh the workspace we can use PowerBuilder IDE to refresh the PBLs without user intervention. pb190.exe already has /deploy option to deploy the workspace, the same way you guys can provide /refresh option to refresh the workspace. https://docs.appeon.com/pb2019r2/pbug/ch01s09.html. I think this is something that you guys can provide very easily so that we don't have to wait for too long to have this feature. Though the permanent solution would be to fix this in orcascript by changing the existing behaviour or adding a new option in orcascript.
	
2. The other approach suggested by Bruce Armstrong in his 2 tutorials.
https://community.appeon.com/index.php/articles-blogs/tutorials-articles/2-powerbuilder/175-continuous-integration-with-powerbuilder-2017-bonobo-git-and-jenkins
https://community.appeon.com/index.php/articles-blogs/tutorials-articles/2-powerbuilder/178-revisiting-continuous-integration-with-powerbuilder-2017-bonobo-git-and-jenkins
This approach also assumes that you will be bootstrapping the pbls. So again that takes up too much time. The main difference between the first official approach and Bruce's approach is: The second approach just uses orcascript to bootstrap PBLs and then uses pbc190 to create the exe. In my testing, it is slower than using the first approach. Because both the approaches use scc refresh target "3pass", since in the first approach everything is already compiled, it is faster to just generate PBDs and exe from the orcascript itself instead of using PowerBuilder compiler for that which would again compile everything.

3. PowerBuilder Compiler
While moving to PowerBuilder 2019 R2, the 2 features we were most excited about were the Git integration and PowerBuilder Compiler. We thought using PowerBuilder Compiler would be so much easier in automating the build process. But as far as I know, there is no way to use the PowerBuilder compiler directly when you are using the Git integration. If you want to use the PowerBuilder compiler you first have to create PBLs using orcascript as mentioned in approach #2 and as mentioned in approach #2, there is no real benefit we are getting from using PowerBuilder Compiler in the second step. So it makes the standalone PowerBuilder Compiler pretty useless when you are using the new Git Integration. 

I have been trying to figure this whole thing out for a week but didn't find any acceptable approach which would let us automate the build process without increasing the build time too much. I can understand that the build process would never be as fast as if we were directly uploading pbls instead of source files. So I don't expect the build time to stay the same, but a 50-minute build process stretching to 2.5 to 3 hours is just unacceptable. I guess the only missing link in this whole approach is a way to refresh pbls from commandline as we can do with IDE. 

Let me know if there is anything that I have overlooked and if there is some other way or some other workaround that we can use to automate the build process without increasing the build time too much.
OS:
All
Platform:
32-bit
Database Type:
Database Version: